Wednesday, 2016-08-17

*** svenkat has joined #openstack-powervm00:08
*** thorst has joined #openstack-powervm00:24
*** thorst has quit IRC00:31
*** svenkat has quit IRC00:43
*** svenkat has joined #openstack-powervm00:43
*** thorst has joined #openstack-powervm00:45
*** seroyer has joined #openstack-powervm00:59
*** svenkat has quit IRC01:30
*** esberglu has joined #openstack-powervm01:35
*** esberglu has quit IRC01:49
*** thorst has quit IRC02:06
*** thorst has joined #openstack-powervm02:06
*** thorst has quit IRC02:15
*** thorst has joined #openstack-powervm02:40
*** thorst has quit IRC02:40
*** thorst has joined #openstack-powervm02:41
*** thorst has quit IRC02:49
*** tjakobs has joined #openstack-powervm03:08
*** thorst has joined #openstack-powervm03:48
*** thorst has quit IRC03:56
*** seroyer has quit IRC04:10
*** thorst has joined #openstack-powervm04:55
*** thorst has quit IRC05:01
*** tjakobs has quit IRC05:17 proposed openstack/nova-powervm: Plug updated VIFs
*** thorst has joined #openstack-powervm06:00
*** thorst has quit IRC06:06
*** thorst has joined #openstack-powervm07:03
*** thorst has quit IRC07:11
*** thorst has joined #openstack-powervm07:29
*** thorst has quit IRC07:34
*** thorst has joined #openstack-powervm08:31
*** k0da has joined #openstack-powervm08:37
*** thorst has quit IRC08:38
*** thorst has joined #openstack-powervm09:37
*** kotra03 has joined #openstack-powervm09:42
*** thorst has quit IRC09:44
*** openstack has joined #openstack-powervm10:17
*** kotra03 has quit IRC10:20
*** kotra03_ has quit IRC10:21
*** kotra03 has joined #openstack-powervm10:23
*** thorst has joined #openstack-powervm10:42
*** thorst has quit IRC10:48
*** smatzek has joined #openstack-powervm11:09
*** thorst has joined #openstack-powervm11:17
*** thorst_ has joined #openstack-powervm11:21
*** thorst has quit IRC11:23
openstackgerritDrew Thorstensen (thorst) proposed openstack/nova-powervm: Wrap console failure message
*** madhaviy has joined #openstack-powervm12:19
*** seroyer has joined #openstack-powervm12:28
thorst_kotra03: is this one fix committed?
openstackLaunchpad bug 1602759 in nova-powervm "get_vm_qp in throws an exception if the instance no longer exists" [Undecided,In progress] - Assigned to Ravi Kumar Kota (ravkota3)12:41
*** kotra03 has quit IRC12:42
*** thorst_ has quit IRC12:50
*** apearson has joined #openstack-powervm12:54
*** apearson has quit IRC12:54
*** apearson has joined #openstack-powervm12:54
*** kylek3h_ has quit IRC12:55
*** burgerk has joined #openstack-powervm12:57
*** tblakes has joined #openstack-powervm12:59
*** mdrabe has joined #openstack-powervm13:08
*** kotra03 has joined #openstack-powervm13:22
*** kylek3h has joined #openstack-powervm13:26
*** edmondsw has joined #openstack-powervm13:37
*** seroyer has quit IRC13:40
*** seroyer has joined #openstack-powervm13:54
*** esberglu has joined #openstack-powervm14:05
*** tjakobs has joined #openstack-powervm14:20
*** mdrabe has quit IRC14:24
*** mdrabe has joined #openstack-powervm14:33
*** openstackgerrit has quit IRC15:03
*** openstackgerrit has joined #openstack-powervm15:04
*** k0da has quit IRC15:25
*** mdrabe has quit IRC15:25
*** mdrabe has joined #openstack-powervm15:25
efriedadreznec, trying to stack (still), now hanging here:15:36
efriedopenstack --os-region-name=RegionOne volume type create --property volume_backend_name=lvmdriver-1 lvmdriver-115:36
efriedAny ideas?  Disable a cinder something-or-other?15:36
adreznecDisable c-vol15:41
efriedadreznec, thanks.15:43
efriedadreznec, well, that got past the hang.  But I don't think this is what we had in mind:16:00
efried2016-08-17 15:57:07.432 | +lib/cinder:create_volume_types:564        openstack --os-region-name=RegionOne volume type create --property volume_backend_name=lvmdriver-1 lvmdriver-116:00
efried2016-08-17 15:57:13.256 | An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-7e625f36-b23a-49d5-9955-4a4cc050e5b1)16:00
*** thorst_ has joined #openstack-powervm16:01
efriedadreznec, do I need cinder at all?16:08
thorst_efried: what are you doing/16:09
efriedthorst_, still trying to stack.16:09
thorst_you generally don't need cinder16:09
thorst_no cinder, no ceilometer really16:10
thorst_for sr-iov bits16:10
efriedHow do I whack those completely?16:10
thorst_disable_service c-api c-vol16:10
thorst_and ceilometer, you'd have to explicitly enable.16:10
efriedThanks, trying...16:11
openstackgerritDrew Thorstensen (thorst) proposed openstack/networking-powervm: Fix bridge mapping change
thorst_efried: While you're in stack hell, want to give that a quick pass?16:14
thorst_and if you get more time...
thorst_adreznec: looks like merged...16:16
thorst_efried: this one too:
thorst_clearly, I'm trying to get my queue shrunk and these bugs closed16:19
efriedthorst_, ack16:20
thorst_thx dude16:20
efriedthorst_, think I'm caught up.16:27
efriedthorst_, mdrabe, are you guys greatly preferring @property duplication vs. nestedprop?16:28
efriedthorst_, something else needs to be disabled?16:29
efried2016-08-17 16:23:00.254 | +lib/cinder:start_cinder:523               die 523 'c-api did not start'16:29
efrieddisable_service cinder?16:30
thorst_esberglu: we don't enable cinder in the CI do we?16:30
openstackgerritMerged openstack/nova-powervm: Plug updated VIFs
efriedthorst_, maybe c-sch?16:31
thorst_efried: yeah16:31
thorst_forgot about that.16:31
efriedthorst_, mdrabe, are you guys greatly preferring @property duplication vs. nestedprop?16:32
thorst_I prefer duplication personally...I know it sux at a technical level (duplication, gross!).  But I know we've got a fair number of people coming on board trying to use the API direct16:33
thorst_so I'm trying to put that lens on.  If I'm learning python and looking at pypowervm...16:33
thorst_now if you don't think its that rough (and I think the later iterations are much cleaner)...then I can be swayed16:33
efriedI certainly don't think it's the most confusing thing about our wrapper classes.16:34
thorst_lol - not a compelling argument!16:34
efriedAnd with them split out line-by-line and documented, I feel like it's pretty readable.16:34
thorst_line by line...made more sense16:35
esbergluefried: What system are you on? I can take a look at your conf.16:36
efriedesberglu, thanks, will slack it to ya.16:36
esbergluthorst: We have cinder on the controller16:36
efriedesberglu, reason I disabled cinder was because this was hanging:16:38
efriedopenstack --os-region-name=RegionOne volume type create --property volume_backend_name=lvmdriver-1 lvmdriver-116:38
madhaviy@esberglu I have made some changes under , but some tests are failing in CI. which are unrelated to my changes16:39
mdrabeefried: I like the dup16:41
efriedmdrabe, even having looked at the latest rev of the other?16:45
mdrabeefried: yea... sort of. Tbh I'm somewhat on the fence. If you put a gun to my head I'd have to say the duplication, it's more readable and if you account for the nestedprop util about the same LOC16:52
openstackgerritDrew Thorstensen (thorst) proposed openstack/networking-powervm: Fix bridge mapping change
*** tblakes_ has joined #openstack-powervm16:54
*** svenkat has joined #openstack-powervm16:54
*** tblakes has quit IRC16:57
*** tblakes_ is now known as tblakes16:57
adreznecthorst_: Forgot to put it in my email, but ceilometer-powervm already has docs:
svenkatthorst_ : regarding, I will patch a live environment16:59
thorst_adreznec: k17:01
thorst_svenkat: k17:01
svenkatthorst_ : agent is now coming up17:04
thorst_svenkat: cool17:05
thorst_mdrabe efried: time to settle the whole pypowervm debate?  Move forward with 3764?17:06
adreznecPatch up at for that17:06
efriedthorst_, mdrabe: Yeah, I would rather have the "extra" LOC in a util lib - which people are less likely to read - than cluttering up the wrappers.17:06
thorst_efried: that's not 3764 though?17:07
mdrabeefried: I would say nestedprop if there were several more nestedprops... might even argue for one of the earlier change sets17:07
thorst_3764 just hides VNICDetails17:07
thorst_well, the problem I have with nestedprops is that it is just a property, but it looks entirely different.17:08
thorst_but its actually the same thing17:08
thorst_pretty much17:08
mdrabeYea it's a little wtfish17:08
efriedThey both hide VNICDetails.  3746 does it (and others) with nestedprops.  3764 does it with duplication.17:08
thorst_but at the same time...I get it?17:08
mdrabe(I really hate these change set numbers) efried: There's other places nestedprop _could_ be used right?17:10
efriedmdrabe, yes.  The nestedprop change set includes using them in two other wrapper classes, greatly reducing those classes' LOC counts.17:11
mdrabeIs that all the cases? I mean there could be even others right? Would just need some investigating17:12
thorst_I think SEA and VEA.17:12
efriedmdrabe, yeah.17:12
thorst_efried: maybe we should put this to the test17:13
thorst_get some third party besides mdrabe and I to look at it17:13
thorst_who has used wrappers17:13
mdrabeNow that I know what this nestedprop thing is exactly I'm starting to think it's not so bad...17:13
thorst_and see if it makes sense.17:13
thorst_with just the comment as is?17:13
mdrabeYea my perspective is no longer fresh17:14
thorst_lmtaylor would be a good one to take a peak at it...17:15
efriedI'll give her a buzz.17:15
efriedI will be objective, I promise.17:15
thorst_efried: OK - don't sway her opinion17:15
*** madhaviy has quit IRC17:31
adreznecefried: thorst_ Have either of you ever seen an issue with pypowervm util pyasn1 modules? E.g.17:41
adreznec2016-08-17 12:40:23.977 141411 ERROR nova.virt.driver     from pyasn1_modules import rfc245917:41
adreznec2016-08-17 12:40:23.977 141411 ERROR nova.virt.driver ImportError: No module named pyasn1_modules17:41
thorst_well, I think something similar17:41
adreznecI verified that it's installed... and tried reinstalling through both pip and apt-get17:41
thorst_try to sudo apt-get remove that17:41
adreznecBut it didn't solve it17:41
thorst_and then install only via pip17:41
openstackgerritDrew Thorstensen (thorst) proposed openstack/nova-powervm: Wrap console failure message
adreznecSo uh17:43
adreznecpypowervm drags them back in as dependencies17:43
thorst_but via pip?17:43
adreznecI guess I could pip install pypowervm...17:44
adreznecand reinstall pvm-cli without deps...17:44
adreznecDo we know what actually causes it?17:45
*** adi___ has quit IRC17:46
*** dwayne has joined #openstack-powervm17:47
adreznecI think our requirements.txt is wrong for pypowervm17:49
adreznecWe're looking for pyasn1_modules, but it appears to really be named pyasn1-modules17:49
*** adi___ has joined #openstack-powervm17:51
*** apearson has quit IRC18:06
*** apearson has joined #openstack-powervm18:09
*** kotra03 has quit IRC18:31
efriedthorst_, mdrabe: I concede that the version with the duplication, though inelegant, is more readable/consumable by the developer.  I hate to give up nestedprop, because it's cool and sexy and was difficult to get right, but it's the better choice.18:55
*** apearson has quit IRC19:01
*** apearson has joined #openstack-powervm19:03
efriedadreznec, thorst_ - now stack finishes "successfully", but rabbitmq-server is down, so nothing works.19:22
efriedIf I unstack, I can get rabbitmq-server to start; but while stacked, it won't.19:22
adreznecefried: What's wrong with it? OOM issue or something?19:26
efriedadreznec, I can't tell.  There's nothing useful in the service status or the logs that I can see - but I'm doubtless looking in the wrong place.19:27
*** edmondsw has quit IRC19:28
*** apearson has quit IRC19:29
*** apearson has joined #openstack-powervm19:30
thorst_adreznec: sde scrum19:31
thorst_efried: yeah...look at the memory.  And the process ctl to see why it died19:32
efriedthorst_, "look at the memory" - like via "free"?  And where is "process ctl"?19:32
thorst_when a process dies, there is some command to look at the logs for that process19:33
thorst_well, not process but 'service'19:33
thorst_systemctl status rabbitmq-server19:33
thorst_or something of that sort.19:33
efriedoh, yeah, that stuff is not helpful at all.19:33
efried"failed to start" kind of thing.19:33
adreznecthorst_: Bob is running it19:35
adreznecI'm busy with rax prep19:35
thorst_yeah, he dialed in19:36
efriedneo@ip9-47-69-149:/opt/stack$ free -m19:36
efried              total        used        free      shared  buff/cache   available19:36
efriedMem:           2454        1743         176           0         534          8019:36
efriedSwap:          3903        3084         81919:36
efriedAnything worrisome in there?19:36
thorst_this is AIO?19:37
thorst_yeah, that's way too much swap19:37
thorst_bump your NL up to 8 GB19:37
thorst_AIO == 8 GB min19:37
thorst_and you're doing the whole smt-off thing so devstack doesn't go nuts with threads?19:39
*** k0da has joined #openstack-powervm19:43
-openstackstatus- NOTICE: The volume for filled up rather suddenly, causing a number of jobs to fail with a POST_FAILURE result and no logs; we're manually expiring some logs now to buy breathing room, but any changes which hit that in the past few minutes will need to be rechecked and/or approved again19:45
adreznecthorst_: tjakobs I think wait_for_vioses is still busted in pypowervm if Novalink is the IO host rather than VIOS19:46
thorst_adreznec: hmm...I thought for SURE we handled that19:46
adreznecI just hit it19:46
adreznecAnd commenting out pvm_par.validate_vios_ready(self.adapter) from let me past immediately19:47
adreznecand the driver loaded fine19:47
thorst_adreznec: lets debug19:47
adreznecnova-powervm and pypowervm are both installed from today19:47
thorst_in 2.5 hours19:47
adreznecThat method in pypowervm has no checks for the management partition that I can tell19:48
thorst_I don't belieb you19:48
adreznecAh crap19:49
adreznecFWIW I installed pypowervm from github here19:49
adreznecMaybe syncing to develop isn't working?19:49
adreznecFrom morpheus19:49
tjakobsi just looked at the github for pypowervm, develop branch. looks like its there19:51
adreznecYeah, let me look locally19:51
adreznecAh I wonder if it installed from master19:52
adreznecSince this is inside the OSA venv19:52
adreznecI bet that's it19:53
adreznecI'll have to verify later, no time now19:54
thorst_adreznec: the final countdown!19:58
openstackgerritMerged openstack/networking-powervm: Fix bridge mapping change
openstackgerritEric Berglund proposed openstack/nova-powervm: DNM: ci check
efriedadreznec, thorst_: What version of pypowervm is in pip?20:57
*** edmondsw has joined #openstack-powervm20:58
*** smatzek has quit IRC20:58
adreznecefried: Something old20:58
adreznecIdk what exactly20:59
efriedThat's bad, right?20:59
adreznecWe're not publishing with an automated job today20:59
adreznecWell, we don't have anyone actively pulling from it iirc...20:59
adreznecBut yes20:59
adreznecIt's bad20:59
efriedadreznec, I found someone actively pulling from it, apparently.20:59
thorst_we're trying to figure out what version to set it to20:59
thorst_esberglu did some testing20:59
thorst_should it be ?20:59
thorst_seems like it CAN be that...21:00
adreznecI guess if it can we should just push that using twine for now21:00
adreznecAnd talk to Dom about getting a job in place21:00
efriedWhat about folks actively wanting to develop against the latest develop branch?21:01
efriedShould they just clone the develop branch from github and pip install . it?21:01
thorst_efried: unfortunately that's a bit tough...we recommend development be against a stable (release) branch21:02
efriedthorst_, okay, but what about a consumer that needs the very latest fixes?21:03
efriedand capabilities21:03
thorst_capabilities aren't there until we release...21:03
efriedthat we haven't released in an officially-tagged version yet21:03
thorst_fixes would go in a hotfix version21:03
thorst_I think there is one coming...
adreznecWe can't publish develop to pypi21:05
adreznecThat's actively discouraged21:05
*** mdrabe has quit IRC21:25
*** mdrabe has joined #openstack-powervm21:26
*** tblakes has quit IRC21:46
*** tjakobs has quit IRC21:49
*** svenkat has quit IRC21:50
*** thorst_ has quit IRC21:54
*** mdrabe has quit IRC22:02
*** apearson has quit IRC22:08
*** esberglu has quit IRC22:14
*** tjakobs has joined #openstack-powervm22:21
*** burgerk has quit IRC22:25
*** tsjakobs has joined #openstack-powervm22:28
*** kylek3h has quit IRC22:28
*** tjakobs has quit IRC22:28
*** seroyer has quit IRC22:48
*** k0da has quit IRC22:52
*** tsjakobs has quit IRC22:55

Generated by 2.14.0 by Marius Gedminas - find it at!