Wednesday, 2016-10-05

*** seroyer has quit IRC00:47
*** thorst has quit IRC00:58
*** thorst has joined #openstack-powervm01:02
*** esberglu has quit IRC01:03
*** thorst has quit IRC01:08
*** thorst has joined #openstack-powervm01:15
*** seroyer has joined #openstack-powervm01:17
*** thorst has quit IRC01:20
*** thorst has joined #openstack-powervm01:26
*** thorst has quit IRC01:29
*** jwcroppe has joined #openstack-powervm02:04
*** thorst has joined #openstack-powervm02:50
*** thorst has quit IRC02:54
*** thorst has joined #openstack-powervm02:54
*** thorst has quit IRC02:55
*** seroyer has quit IRC03:04
*** thorst has joined #openstack-powervm03:05
*** thorst has quit IRC03:11
*** tjakobs has joined #openstack-powervm03:43
*** tjakobs has quit IRC03:48
*** shyama has joined #openstack-powervm03:59
*** jwcroppe has quit IRC05:09
*** thorst has joined #openstack-powervm05:11
*** thorst has quit IRC05:16
*** adi___ has quit IRC05:41
*** adi___ has joined #openstack-powervm05:46
*** jwcroppe has joined #openstack-powervm05:58
*** jwcroppe has quit IRC06:04
*** kotra03 has joined #openstack-powervm06:21
*** kotra03 has quit IRC06:26
*** jwcroppe has joined #openstack-powervm06:56
*** jwcroppe has quit IRC07:02
*** AlexeyAbashkin has quit IRC07:07
*** k0da has joined #openstack-powervm07:43
*** AlexeyAbashkin has joined #openstack-powervm07:46
*** viclarson has joined #openstack-powervm08:04
*** jwcroppe has joined #openstack-powervm08:42
*** jwcroppe has quit IRC08:51
*** thorst has joined #openstack-powervm09:13
*** thorst has quit IRC09:19
*** viclarson has quit IRC09:25
*** jwcroppe has joined #openstack-powervm10:32
*** jwcroppe has quit IRC10:38
*** viclarson has joined #openstack-powervm10:42
*** viclarson has quit IRC11:07
*** seroyer has joined #openstack-powervm11:26
*** thorst has joined #openstack-powervm11:51
*** viclarson has joined #openstack-powervm12:16
*** seroyer has quit IRC12:30
viclarsonhow vm d obtain ip in novalink?12:34
viclarsonvia dhcp or via config injection?12:35
*** apearson has joined #openstack-powervm12:48
*** seroyer has joined #openstack-powervm12:51
*** kylek3h has joined #openstack-powervm12:52
*** tblakes has joined #openstack-powervm12:55
*** edmondsw has joined #openstack-powervm12:59
*** mdrabe has joined #openstack-powervm13:09
*** tjakobs has joined #openstack-powervm13:24
*** esberglu has joined #openstack-powervm13:39
*** kriskend has joined #openstack-powervm13:57
viclarsonmodified interfaces.template14:13
viclarson# The loopback network interface14:13
viclarsonauto lo14:13
viclarsoniface lo inet loopback14:13
viclarsonauto eth014:13
viclarsoniface eth0 inet dhcp14:13
viclarsonauto eth114:13
viclarsoniface eth1 inet manual14:13
viclarsoni think it's enough14:13
viclarsonotherwise i had eth0 non-configured14:14
efriedthorst, re your review of - got a sec to discuss?14:24
efriedI like the way you're thinking.  My only concern is with the heal/optimize or reset events, we would be hammering neutron for all those device details.  Then again, I think we're doing that anyway.  How expensive are those RPC calls?14:25
*** esberglu has quit IRC14:38
*** edmondsw has quit IRC14:38
*** k0da has quit IRC14:38
*** esberglu has joined #openstack-powervm14:39
*** k0da has joined #openstack-powervm14:39
*** edmondsw has joined #openstack-powervm14:39
*** burgerk has joined #openstack-powervm14:40
*** tjakobs has quit IRC14:41
efriedthorst, ready to merge ?14:43
esbergluJust wanted to make sure that everyone is aware of the newton coordinated release date of October 6th14:48
efriedWhat does that mean?14:49
efriedAnyone know how to clean up /opt/stack/logs?  Everything I've tried (removing the date-stamped file, truncating the date-stamped file) results in the logs no longer being written, even after restarting the respective service.14:52
esbergluEveryone trying to stay in line with the openstack release schedule will be putting out their newton release14:53
*** tjakobs has joined #openstack-powervm15:05
*** k0da has quit IRC15:24
*** shyama has quit IRC15:27
efriedesberglu, insight into these failures?
esbergluStill haven’t figured out the multiple networks found error. Just hitting that15:37
thorstefried: sorry...'stuff' happened.  I think we're already doing those RPC calls15:46
thorstbtw - I was talking with btang, and those rpc loops in the code today...may full on elevate this to a bug.  The calls are spending a ton of CPU in that tight loop15:47
thorsthovering around 70% on a core15:47
efriedthorst, any idea which part is spiking the CPU?15:50
efriedIf we're only calling every 2s, we shouldn't see a continuous hover.15:51
efriedI would believe the event listener could do that, though.15:51
openstackgerritMerged openstack/networking-powervm: Time out port update waiting for vif plug
thorstefried: I believe its due to the vNIC queries?15:54
efriedoh, yeah, that'd do it.15:54
efriedSo this wasn't a problem for the SEA driver?15:54
thorstefried: nope15:55
thorstviclarson: Sorry - just saw your question.  I use DHCP for mine...  My actual /etc/network/interfaces that lands on the VM is similar to yours15:56
thorstexcept eth1 has: iface eth1 inet6 auto15:56
thorstthat way it gets the Link Local IPv6 address15:56
*** mdrabe has quit IRC15:57
*** mdrabe has joined #openstack-powervm16:01
*** seroyer has quit IRC16:04
viclarsonthorst why we need eth1 configured?16:06
thorstviclarson: That allows for the Secure RMC connection between the hypervisor and the VM.  It allows for operations like live migration, adding/removing CPUs live, etc...16:07
thorstwe call it 'DLPAR' operations.16:08
viclarsonmy eth1 configured as 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 100016:12
viclarson    link/ether 56:1d:fc:63:63:3f brd ff:ff:ff:ff:ff:ff16:12
viclarson    inet6 fe80::541d:fcff:fe63:633f/64 scope link16:12
viclarson       valid_lft forever preferred_lft forever16:12
viclarsonis it enough?16:12
*** seroyer has joined #openstack-powervm16:17
thorstshould be...if you're using the image we provided it should already have the RMC software packages installed16:20
thorstso I think you're good.16:20
thorstviclarson: ^^16:20
*** mdrabe has quit IRC16:31
*** mdrabe has joined #openstack-powervm16:31
*** shyama has joined #openstack-powervm16:45
*** svenkat has joined #openstack-powervm17:01
*** tblakes has quit IRC17:04
thorstefried mdrabe: Can you make sure I addressed your concerns here:
thorstbefore I ask the nova core team to review, I'd like to make sure we're all +1 on it.17:20
efriedthorst, couple more comments from edmondsw17:23
thorstefried: Yeah, but I don't want to flood the nova channel with minor updates17:23
efriedI understand.17:23
thorstI'd rather collect them all and do one change....17:23
thorstso if you and mdrabe are +1, then I'll get edmondsw done and do a new one17:23
viclarsonafter creating vm with flavor hd 20GB i see17:34
viclarsonroot@ubuntu:/home/ubuntu# mount17:34
efriedthorst, reviewed.17:34
viclarsonmounted /dev/sdb2 on / type ext4 (rw,errors=remount-ro)17:35
viclarsonmounted /dev/sr0 on /tmp/tmpzmGh0G type iso9660 (ro,sync,relatime)17:35
viclarsonand /dev/sda1 (20GB) formatted to ext4fs but not mounted17:36
viclarsonsomething went wrong?17:36
viclarsonalso /tmp/tmp... doesnt exist17:36
thorstviclarson: did you change the partition tables at all?17:36
viclarsonwhat is expected configuration?17:38
thorstso PowerVM has a boot partition.  Should be in any image that loads.  Its about 17 meg or so.  Then there is the actual data partition.  So you should typically see a /dev/sda1 (boot partition - 17 meg) and then /dev/sdb2 (the remainin 20 gigs).17:40
viclarson(parted) print all17:41
viclarsonModel: IBM 3303 NVDISK (scsi)17:41
viclarsonDisk /dev/sda: 21.5GB17:41
viclarsonSector size (logical/physical): 512B/512B17:41
viclarsonPartition Table: gpt17:41
viclarsonNumber  Start   End     Size    File system  Name  Flags17:41
viclarson 1      1049kB  8389kB  7340kB                     prep17:41
viclarson 2      8389kB  21.5GB  21.5GB  ext417:41
viclarsonfirst drive17:41
viclarsonModel: IBM 3303 NVDISK (scsi)17:42
viclarsonDisk /dev/sdb: 21.5GB17:42
viclarsonSector size (logical/physical): 512B/512B17:42
viclarsonPartition Table: gpt17:42
viclarsonNumber  Start   End     Size    File system  Name  Flags17:42
viclarson 1      1049kB  8389kB  7340kB                     prep17:42
viclarson 2      8389kB  2146MB  2138MB  ext417:42
viclarsonabd second17:42
viclarsonand second17:42
viclarsoni requested boot from image, no additional volumes17:43
viclarsonis it some intermediate state?17:50
viclarsonthorst: ^^18:01
thorstviclarson: no, that 7 meg thing looks right to me18:13
*** tblakes has joined #openstack-powervm18:13
thorstahh, I know what the problem is18:13
thorstI think I do at least....18:13
thorstthis is dual VIOS...its SSP, right?18:14
thorstefried: Could we be doing some sort of dual VIOS thing with the SSP load up?18:15
thorstI know that the image I provided didn't have the MPIO drivers loaded into them18:15
thorstso it was meant for single VIOS...  I'm not sure how to add MPIO after the fact.18:15
viclarsondoesnt matter18:17
thorstmultipath means that there are multiple paths to the same disk.18:17
thorstand there have to be multi path device drivers loaded in the OS18:17
thorstso that the OS, looking through two different busses (ex. two different vSCSIs) recognizes it as the same disk18:17
*** thorst is now known as thorst_afk18:17
thorst_afkmpio == multipath, yeah18:17
thorst_afkmulti path i/o18:17
viclarsoncould you plz provide image which can work18:18
viclarsonperhaps it's time for aix18:18
mdrabethorst: About the blueprint, is the goal to use existing os-vif plugins like OVS, meaning we wouldn't support SEA just yet?18:45
thorst_afkmdrabe: yep18:45
*** thorst_afk is now known as thorst_18:45
thorst_mdrabe: do you know if we have a linux image with MPIO that is ubuntu laying around?18:45
mdrabethorst_: Not that I'm aware unfortunately18:46
thorst_mdrabe: OK - I'll check with some of the testers18:47
efriedthorst_ - "dual VIOS thing with the SSP load up" - what do you mean?18:50
thorst_efried: Sorry - you validated that the disk driver for SSP works with dual VIOS right?18:51
efriedShoot, I think so.  Like 18 months ago when we first developed it.18:51
efriedI mean, other than that problem pvc recently saw where mappings supposedly needed to be serialized.18:52
efriedWhat's the problem we're seeing?18:52
*** k0da has joined #openstack-powervm19:00
efriedthorst_, help me brainstorm this a sec...19:05
efriedWe're operating on the assumption that the neutron port update is synchronous, right?19:06
efriedThat is, the compute manager grabs the port and tells neutron to send the port update; and waits for that guy to come back before it proceeds with the rest of the operation (spawn or attach).19:06
efriedWith your review suggestions, I'm ripping the whole neutron side of the ProvisionRequest queues out, which leaves port_update empty.19:08
efriedI'm thinking we add in a bridge mapping update there.19:08
efriedI.e. refresh the agent's view of the pport label:physloc mappings *and* send that back to neutron via the _report_state RPC.19:08
efriedThat way, by the time we get to the vif driver, the neutron view of the agent is "guaranteed" to be up to date - at least with whatever the view was when the port was assigned, which is what we want.19:09
efriedThus eliminating the potential staleness window we were seeing.19:10
*** shyama has quit IRC19:13
thorst_efried: Just wondering if we handled MPIO19:31
thorst_looking at other q's...sorry, was talking SR-IOV19:31
thorst_efried: its not 100% synchronous.  The port update happens very early on...19:32
thorst_well, what port update are we talking about?19:32
efriedthorst_, the one we register as an RPC callback.19:41
efriedThe one from which we were previously queueing a "neutron port request".19:41
efriedWe were kinda counting on that coming in first anyway with this whole queuing thing.  We would discard any custom events that came in if they didn't have a corresponding neutron req already registered.19:42
thorst_efried: yeah, that should be like an instant back19:52
efriedInstant enough for us to do a managed system GET?19:53
thorst_I'm basically saying if there are custom events, and the data within them are good, that's all we really need19:53
efriedYeah, this is a separate discussion.19:53
thorst_efried: I don't follow that second q19:53
efriedThis is for getting the proper picture of pport label:physloc mappings back to the mech driver, so by the time it feeds those back to the vif driver, they're accurate.19:54
thorst_I see what you're saying19:54
thorst_yeah yeah...I see.19:54
thorst_hmm...I guess from an SR-IOV side, we could try that19:54
thorst_but I'd only want to do that if the request came in was a SR-IOV request.19:54
efriedYeah, there's no equivalent in SEA.19:55
efriedThe SEA agent's port_update is now a no-op.19:55
thorst_keep in mind a port update could come in for like anything...a QoS policy change for instance19:55
thorst_but I'm still OK with that19:56
thorst_managed system get is pretty quick I though19:56
efriedyes, it is.19:56
*** svenkat has quit IRC20:59
openstackgerritEric Fried proposed openstack/networking-powervm: Refactor, consolidate, and clean up agents
openstackgerritEric Fried proposed openstack/networking-powervm: Refactor, consolidate, and clean up agents
efriedthorst_ ^^  Good call.  This cut out a ton.21:17
*** apearson has quit IRC21:18
openstackgerritEric Fried proposed openstack/networking-powervm: Refactor, consolidate, and clean up agents
thorst_efried: you're the one doing the hard work.21:20
thorst_I'm just trying to figure out how to make the I/O not zap out for a few seconds on LPM21:21
*** dwayne has quit IRC21:21
*** tblakes has quit IRC21:22
*** thorst_ has quit IRC21:26
*** thorst_ has joined #openstack-powervm21:26
*** thorst_ has quit IRC21:31
*** kriskend has quit IRC21:45
*** edmondsw has quit IRC21:53
*** mdrabe has quit IRC22:07
*** esberglu has quit IRC22:15
*** seroyer has quit IRC22:23
*** tjakobs has quit IRC22:27
*** burgerk has quit IRC22:33
*** esberglu has joined #openstack-powervm22:48
*** esberglu has quit IRC22:48
*** k0da has quit IRC22:50
*** tjakobs has joined #openstack-powervm23:35
*** dwayne has joined #openstack-powervm23:43

Generated by 2.14.0 by Marius Gedminas - find it at!