IVM and VLAN Tagging

IVM and VLAN Tagging

cggibbo |Nov 8 2011| Comments (9) | Visits (39247)

1

I was working with a customer recently on a Power Blade that was running the Integrated Virtualisation Manager (IVM). They’d installed a VIO partition onto the Blade and had hoped to install a couple of AIX LPARs on the system. However they didn’t get very far.

 

As soon as they attempted to NIM install the LPARs, they would get stuck at trying to ping the NIM master from the client. Basically, the Shared Ethernet Adapter (SEA) was not working properly and none of the LPARs could communicate with the external network. So they asked for some assistance.

 

The Blade server name was Server-8406-71Y-SN06BF99Z. The SEA was configured as ent7.

 

# lsdev -Cc adapter

ent0    Available       Logical Host Ethernet Port (lp-hea)

ent1    Available       Logical Host Ethernet Port (lp-hea)

ent2    Available       Virtual I/O Ethernet Adapter (l-lan)

ent3    Available       Virtual I/O Ethernet Adapter (l-lan)

ent4    Available       Virtual I/O Ethernet Adapter (l-lan)

ent5    Available       Virtual I/O Ethernet Adapter (l-lan)

ent6    Available       EtherChannel / IEEE 802.3ad Link Aggregation

ent7    Available       Shared Ethernet Adapter

ent9    Available       Virtual I/O Ethernet Adapter (l-lan)

fcs0    Available 02-00 8Gb PCIe FC Blade Expansion Card (7710322577107601)

fcs1    Available 02-01 8Gb PCIe FC Blade Expansion Card (7710322577107601)

ibmvmc0 Available       Virtual Management Channel

lhea0   Available       Logical Host Ethernet Adapter (l-hea)

sissas0 Available 01-08 PCI-X266 Planar 3Gb SAS Adapter

usbhc0  Available 00-08 USB Host Controller (33103500)

usbhc1  Available 00-09 USB Host Controller (33103500)

usbhc2  Available 00-0a USB Enhanced Host Controller (3310e000)

vhost0  Available       Virtual SCSI Server Adapter

vsa0    Available       LPAR Virtual Serial Adapter

vts0    Available       Virtual TTY Server Adapter

 

The SEA was configured with Port-VLAN ID (PVID) of 68 without any VLAN tags. This was the root cause of the problem.

 

# lsattr -El ent7

accounting    enabled  Enable per-client accounting of network statistics                 True

ctl_chan               Control Channel adapter for SEA failover                           True

gvrp          no       Enable GARP VLAN Registration Protocol (GVRP)                      True

ha_mode       disabled High Availability Mode                                             True

jumbo_frames  no       Enable Gigabit Ethernet Jumbo Frames                               True

large_receive no       Enable receive TCP segment aggregation                             True

largesend     0        Enable Hardware Transmit TCP Resegmentation                        True

netaddr       0        Address to ping                                                    True

pvid          68       PVID to use for the SEA device                                     True

pvid_adapter  ent9     Default virtual adapter to use for non-VLAN-tagged packets         True

qos_mode      disabled N/A                                                                True

real_adapter  ent6     Physical adapter associated with the SEA                           True

thread        1        Thread mode enabled (1) or disabled (0)                            True

virt_adapters ent9     List of virtual adapters associated with the SEA (comma separated) True

 

$ entstat -all ent8 | grep -i vlan

VLAN Ids :

    VLAN Extract: False

    VLAN tagged filtering mode: Filter according to VLAN permit array

Max number of VLAN IDs per HEA port: 20

    VLAN Extract: False

    VLAN tagged filtering mode: Filter according to VLAN permit array

Max number of VLAN IDs per HEA port: 20

Invalid VLAN ID Packets: 5388

Port VLAN ID:    68

VLAN Tag IDs:    None

 

On the network switch port, the native VLAN (PVID), was configured as 11, with VLAN tag 68 added as an allowed VLAN. If the client LPARs tried to access the network using a PVID of 68, instead of a VLAN TAG of 68, they would get stuck at the switch port i.e. the un-tagged packets for 10.1.68.X via PVID 11 would fail. The packets for 10.1.68.X needed to be tagged with VLAN id 68 in order for the switch to pass the traffic.

 

So the question was, how do we add VLAN tags in the IVM environment? If we’d been using a HMC, then this would be simple to fix. Just add the VLAN tags into the Virtual Ethernet Adapter used by the SEA and we’d be done.

 

We had to use the lshwres and chhwres commands to resolve this one. First we listed the virtual adapters known to the VIO server (IVM). At slot 12, we found our SEA adapter with port_vlan_id set to 68 and addl_vlan_ids set to none.

 

$ lshwres -r virtualio --rsubtype eth --level lpar

lpar_name=06-BF99Z,lpar_id=1,slot_num=3,state=1,ieee_virtual_eth=0,port_vlan_id=1,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=F67D5DA62803

lpar_name=06-BF99Z,lpar_id=1,slot_num=4,state=1,ieee_virtual_eth=0,port_vlan_id=2,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=F67D5DA62804

lpar_name=06-BF99Z,lpar_id=1,slot_num=5,state=1,ieee_virtual_eth=0,port_vlan_id=3,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=F67D5DA62805

lpar_name=06-BF99Z,lpar_id=1,slot_num=6,state=1,ieee_virtual_eth=0,port_vlan_id=4,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=F67D5DA62806

lpar_name=06-BF99Z,lpar_id=1,slot_num=12,state=1,ieee_virtual_eth=0,port_vlan_id=68,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=F67D5DA6280C

lpar_name=aixlpar1,lpar_id=2,slot_num=4,state=1,ieee_virtual_eth=0,port_vlan_id=68,addl_vlan_ids=none,is_trunk=0,trunk_priority=0,is_required=0,mac_addr=F67D5345AD04

 

We needed to change port_vlan_id to 11 and addl_vlan_ids to 68. We also required the ieee_virtual_eth value set to 1.

 

First we removed the existing SEA adapter, as we would not be able to make changes to it while it was “active”. We then removed the adapter from slot 12 and then re-added it, again at slot 12, with port_vlan_id and addl_vlan_ids set to the desired values.

 

$ chhwres -m Server-8406-71Y-SN06BF99Z -p 06-BF99Z -r virtualio --rsubtype eth -s 12 -o r 

$ chhwres -m Server-8406-71Y-SN06BF99Z -p 06-BF99Z -r virtualio --rsubtype eth -s 12 -o a -a "ieee_virtual_eth=1,port_vlan_id=11,addl_vlan_ids=68,is_trunk=1,trunk_priority=1" -d 5

 

$ lshwres -r virtualio --rsubtype eth --level lpar

lpar_name=06-BF99Z,lpar_id=1,slot_num=3,state=1,ieee_virtual_eth=0,port_vlan_id=1,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=F67D5DA62803

lpar_name=06-BF99Z,lpar_id=1,slot_num=4,state=1,ieee_virtual_eth=0,port_vlan_id=2,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=F67D5DA62804

lpar_name=06-BF99Z,lpar_id=1,slot_num=5,state=1,ieee_virtual_eth=0,port_vlan_id=3,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=F67D5DA62805

lpar_name=06-BF99Z,lpar_id=1,slot_num=6,state=1,ieee_virtual_eth=0,port_vlan_id=4,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=F67D5DA62806

lpar_name=06-BF99Z,lpar_id=1,slot_num=12,state=1,ieee_virtual_eth=1,port_vlan_id=11,addl_vlan_ids=68,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=F67D5DA6280C

lpar_name=aixlpar1,lpar_id=2,slot_num=4,state=1,ieee_virtual_eth=0,port_vlan_id=68,addl_vlan_ids=none,is_trunk=0,trunk_priority=0,is_required=0,mac_addr=F67D5345AD04

 

 

Using the mkvdev command we created the SEA again. Then using the entstat command we found that the PVID and VLAN tags had been configured correctly.

 

$ mkvdev -sea ent6 -vadapter ent2 -default ent2 -defaultid 11

ent7 Available

en7

et7

 

$ entstat -all ent7 | grep -i vlan

VLAN Ids :

    VLAN Extract: False

    VLAN tagged filtering mode: Filter according to VLAN permit array

Max number of VLAN IDs per HEA port: 20

    VLAN Extract: False

    VLAN tagged filtering mode: Filter according to VLAN permit array

Max number of VLAN IDs per HEA port: 20

Invalid VLAN ID Packets: 5388

Port VLAN ID:    11

VLAN Tag IDs:    68

 

Once this was done, the client LPARs were able to ping the NIM master. The customer happily started installing AIX onto each of the blades Partitions.

0 (0)
Article Rating (No Votes)
Rate this article
Attachments
There are no attachments for this article.
Comments
There are no comments for this article. Be the first to post a comment.
Full Name
Email Address
Security Code Security Code
Related Articles RSS Feed
LVM: Extend an existing Volume Group by adding a new disk
Viewed 862 times since Sat, Jun 2, 2018
AIX rootvg Mirroring
Viewed 1271 times since Mon, May 21, 2018
How to setup an user in AIX
Viewed 867 times since Mon, May 28, 2018
AIX HOW TO CLONE A ROOTVG USING ALTERNATE DISK INSTALLATION ALTER_DISK_COPY
Viewed 1127 times since Sun, Jun 30, 2019
How to upgrade TL and SP in AIX using alternate disk(alt_disk_copy)
Viewed 916 times since Sun, Jun 30, 2019
AIX disk queue depth tuning for performance
Viewed 392 times since Thu, Jan 16, 2020
List AIX File Systems the Easy Way With the lsvgfs Command
Viewed 556 times since Thu, Sep 20, 2018
Configuration of AIX Fast Connect and SMBFS
Viewed 917 times since Mon, Jun 18, 2018
How To Mirror Your Root Disk On AIX (a.k.a. rootvg)
Viewed 1211 times since Mon, May 21, 2018
How to Maintain a Virtual I/O Server With FBO Part II
Viewed 547 times since Wed, Jun 5, 2019