Tuesday, 2016-05-10

*** Goneri has joined #tripleo00:32
openstackgerritFeng Pan proposed openstack/tripleo-heat-templates: Change ping wait flag  https://review.openstack.org/31434400:32
*** Goneri has quit IRC00:39
*** pabelanger has quit IRC00:42
*** jpena has joined #tripleo00:44
*** mburned_out has joined #tripleo00:44
*** pabelanger has joined #tripleo00:45
*** hewbrocca-afk has joined #tripleo00:46
*** mbound has joined #tripleo00:47
*** mburned_out is now known as mburned00:50
*** mbound has quit IRC00:52
*** dmacpher-afk has joined #tripleo00:52
*** al has quit IRC00:59
*** al has joined #tripleo01:01
*** trozet has quit IRC01:07
*** rhallisey has quit IRC01:48
*** saneax is now known as saneax_AFK01:51
*** yamahata has joined #tripleo02:03
*** ayoung has quit IRC02:17
*** saneax_AFK is now known as saneax02:40
*** coolsvap has joined #tripleo02:41
*** rajinir has joined #tripleo02:44
*** jokke_ has quit IRC02:45
*** jokke_ has joined #tripleo02:45
*** yuanying has quit IRC02:51
*** stendulker has joined #tripleo03:04
*** shivrao has quit IRC03:12
*** akshai has joined #tripleo03:18
*** masco has joined #tripleo03:22
*** links has joined #tripleo03:29
*** ebalduf has joined #tripleo03:34
*** ccamacho has quit IRC03:36
*** ccamacho has joined #tripleo03:39
*** ayoung has joined #tripleo03:42
*** panda has quit IRC03:42
*** panda has joined #tripleo03:42
*** shivrao has joined #tripleo03:43
*** shivrao has quit IRC03:43
*** tzumainn has quit IRC03:48
*** yuanying has joined #tripleo03:48
*** sirushti has quit IRC03:57
*** sirushti has joined #tripleo04:06
*** saneax is now known as saneax_AFK04:11
*** stendulker_ has joined #tripleo04:19
*** stendulker has quit IRC04:23
*** rlandy has quit IRC04:24
*** stendulker has joined #tripleo04:27
*** stendulker_ has quit IRC04:31
*** ramishra has joined #tripleo04:33
*** veteran has joined #tripleo04:42
*** shivrao has joined #tripleo04:43
*** shivrao_ has joined #tripleo04:43
*** rajinir has quit IRC04:46
*** saneax_AFK is now known as saneax04:47
*** shivrao has quit IRC04:47
*** shivrao_ is now known as shivrao04:47
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Create keystone endpoints in keystone profile  https://review.openstack.org/30412504:54
*** trozet has joined #tripleo04:56
*** fragatina has quit IRC04:59
*** akshai has quit IRC05:00
*** trozet has quit IRC05:16
*** fragatina has joined #tripleo05:17
*** fragatina has quit IRC05:22
*** coolsvap has quit IRC05:25
*** itamarl has joined #tripleo05:31
*** fragatina has joined #tripleo05:33
*** fragatina has quit IRC05:33
*** fragatina has joined #tripleo05:34
*** saneax is now known as saneax_AFK05:37
openstackgerritMerged openstack/tripleo-ui: Remove module cache files from git  https://review.openstack.org/31354605:37
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Tracking nodes operations in Nodes Table  https://review.openstack.org/31423305:51
*** d0ugal has joined #tripleo05:52
*** itamarl has quit IRC05:53
*** oshvartz has joined #tripleo06:05
*** itamarl has joined #tripleo06:06
*** apetrich has joined #tripleo06:24
*** rcernin has joined #tripleo06:26
*** itamarl has quit IRC06:26
*** apetrich has quit IRC06:27
*** apetrich has joined #tripleo06:28
*** dmacpher-afk has quit IRC06:31
*** bvandenh has joined #tripleo06:32
*** cwolferh has quit IRC06:34
*** jprovazn has joined #tripleo06:37
*** tremble has joined #tripleo06:38
*** shivrao has quit IRC06:39
*** mcornea has joined #tripleo06:44
*** itamarl has joined #tripleo06:45
*** florianf has joined #tripleo06:46
*** tesseract has joined #tripleo07:03
*** sshnaidm has quit IRC07:03
*** shivrao has joined #tripleo07:07
*** dsariel has joined #tripleo07:11
*** hewbrocca-afk is now known as hewbrocca07:14
*** zoli_gone-proxy is now known as zoliXXL07:15
*** zoliXXL is now known as zoli|wfh07:15
*** d0ugal has quit IRC07:23
*** d0ugal has joined #tripleo07:26
*** d0ugal has quit IRC07:26
*** d0ugal has joined #tripleo07:26
zoli|wfhgood morning07:28
*** paramite has joined #tripleo07:30
*** shivrao has quit IRC07:30
*** mikelk has joined #tripleo07:34
*** olap has joined #tripleo07:36
*** ifarkas has joined #tripleo07:43
*** aufi has joined #tripleo07:48
*** paramite is now known as paramite|afk07:51
*** liverpooler has joined #tripleo07:52
*** shardy has joined #tripleo07:56
*** paramite|afk is now known as paramite07:59
*** jaosorior has joined #tripleo08:01
jaosoriorhey dude08:05
shardy'morning folks08:05
*** gfidente has joined #tripleo08:08
*** SlickNik has quit IRC08:10
*** SlickNik has joined #tripleo08:11
bandinimorning *08:11
*** ohamada has joined #tripleo08:12
*** athomas has joined #tripleo08:14
*** dtantsur|afk is now known as dtantsur08:16
ccamachoGuys, just to recheck is anyone working in a AllNodesServices, like the ControllerServices resource for composable THT?08:17
*** yamahata has quit IRC08:20
*** SlickNik has quit IRC08:22
*** lucas-dinner is now known as lucasagomes08:22
*** SlickNik has joined #tripleo08:22
shardyccamacho: You mean like OS::TripleO::AllNodesExtraConfig ?08:25
shardyccamacho: we'll have to maintain all the existing ExtraConfig interfaces, even if we do implement an alternative method via the composable services model08:26
gfidenteso I opened a LP for the issue which is causing upgrades job to fail08:26
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,Triaged]08:26
gfidentebut I am still investigating why that happens08:26
shardygfidente: nice, thanks for looking into it!08:26
gfidentelooks like computes can't reach keystone and the pingtest vm can't be scheduled08:26
*** devvesa has joined #tripleo08:27
gfidentebut I haven't figured why that is, so I am trying to reproduce locally08:27
shardyccamacho: Oh, do you mean a parameter like AllNodesServices, that puts a service on all roles?08:27
shardypersonally I would prefer we didn't do that, as IMHO it's clearer to just add a service to the list for all roles08:28
shardyand also it's going to be very difficult to do when we enable arbitrary custom roles08:28
shardyThe first priority is adding a $RoleServices parameter for all existing roles, and decomposing all the services on the roles we already have08:30
shardythen we need to enable arbitrary custom roles, which will basically repeat the same pattern for user provided additional roles08:30
*** cmyster has joined #tripleo08:32
*** cmyster has quit IRC08:32
*** cmyster has joined #tripleo08:32
ccamachoshardy, like ControllerServiceChain for the controller services (https://review.openstack.org/#/c/259568/) but for all nodes08:33
shardyccamacho: What advantage would that have, over say adding the service explicitly to each role's list of services?08:35
shardyccamacho: I'm thinking that although that's a slightly more verbose interface, it's more explicit (easier for e.g UI to show what services are running on each role), and also potentially cleaner when the implementation needs to be adapted to support custom user-defined roles (which I'm currently working on based on the jinja template approach we talked about at summit)08:37
*** jistr has joined #tripleo08:37
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo: Add Cinder API/Scheduler/Volume roles  https://review.openstack.org/31093908:39
*** sshnaidm has joined #tripleo08:40
ccamachoshardy, ok, is that I have finished the NTP service with composable roles but, wanted to keep working to land it sooner as it applies to all roles08:44
shardyccamacho: Sure, in that case, I'd suggest writing patches that wire in parameters for all existing roles, just like ControllerServices08:45
shardythat will help because it's a prerequisite to decomposing all the existing roles08:45
shardyand it will enable you to wire in your NTP patch to all roles08:45
shardywe can revisit the AllNodes thing later, but I think we should land the per-role interfaces first08:45
ccamachosure, thanks!08:46
*** derekh has joined #tripleo08:47
shardyccamacho: I added a section for per-role prerequistes there, feel free to update08:49
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Install virtualenv for tempest  https://review.openstack.org/31446208:50
*** anshul has joined #tripleo08:53
ccamachoshardy, nice!08:53
*** anshul is now known as Guest6415308:54
*** itamarl_ has joined #tripleo09:06
*** itamarl has quit IRC09:08
*** itamarl_ is now known as itamarl09:09
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Add timestamps to postci log  https://review.openstack.org/31446809:11
*** ebalduf has quit IRC09:13
*** saneax_AFK is now known as saneax09:20
*** milan has quit IRC09:28
*** chem has joined #tripleo09:29
*** chlong has quit IRC09:30
*** mgould has joined #tripleo09:34
*** gfidente has quit IRC09:40
*** tosky has joined #tripleo09:45
shadowermandre: ping09:53
shardydtantsur: just FYI - I think it was you who complained re the annoying SIGNAL_COMPLETE: Unknown events, this is the fix https://review.openstack.org/#/c/314226/09:58
derekhIs anybody looking into whats up with the upgrades job?09:58
*** coolsvap has joined #tripleo09:58
shardyderekh: I think gfidente is09:58
derekhshardy: ack, will chat to him once he comes online10:00
shardyhe was around earlier, hopeully will show up again soon :)10:01
*** milan has joined #tripleo10:01
openstackgerritMarios Andreou proposed openstack/tripleo-common: Adds a simple 'overcloud_shell' mistral worfklow  https://review.openstack.org/31308410:02
*** jaosorior has quit IRC10:09
*** jaosorior has joined #tripleo10:09
*** ooolpbot has joined #tripleo10:10
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,Triaged]10:10
*** ooolpbot has quit IRC10:10
*** liverpooler has quit IRC10:26
*** liverpooler has joined #tripleo10:26
*** chem has quit IRC10:45
*** rodrigods has quit IRC10:49
*** rodrigods has joined #tripleo10:49
*** chem has joined #tripleo10:49
*** hewbrocca is now known as hewbrocca-afk10:51
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge.  https://review.openstack.org/31071310:53
*** dtantsur is now known as dtantsur|bbl10:55
*** jpena is now known as jpena|lunch10:55
mandreshadower: pong10:56
shadowermandre: so the only validanion not working under Ansible 2 is the mysql file limits one10:57
shadowermandre: I just looked it and unfortunately, the mysql_variables module we use there has incompatible output between 1.9 and 2.010:57
shadowermandre: so instead of hacking around that, I'd rather we just switched to 2.0 wholesale10:58
shadowerwhat do you think?10:58
mandreshadower: seems good to me10:58
shadowercool, I'll send a patch10:59
shadowerof course, if the deployers need 1.9 support, we can add it later, but I'd like 2.0 be the default10:59
mandreexcellent, looking forward to try it out10:59
mandreagreed, 2.0 is a better default11:00
ccamachoderekh, gfidente I pasted some more info about (https://bugs.launchpad.net/tripleo/+bug/1580076) on the ticket11:02
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,Triaged]11:02
derekhccamacho: thanks, The message you posted "Message: No valid host was found. There are not enough hosts available." comes from nova any time it fails to schedule a VM,11:05
*** stendulker has quit IRC11:06
derekhccamacho: it looks like the reason it can't start the VM is because the nova compute node is trying to talk to http://[2001:db8:fd00:1000::10]:5000/v3/auth/tokens but its can't for some reason11:06
*** rhallisey has joined #tripleo11:06
ccamachoderekh: Im trying to reproduce it again (I did that yesterday) to get more nova details on my lab, the thing is that this behavior does not happen all the time, In my case only when creating/deleting the overcloud some times.11:08
derekhccamacho: ok, I'm trying to do the same, hopfully between us we'll figure it out11:08
ccamachosure :)11:09
*** ooolpbot has joined #tripleo11:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,Triaged]11:10
*** jaosorior has quit IRC11:20
*** jaosorior has joined #tripleo11:20
openstackgerritDerek Higgins proposed openstack/tripleo-heat-templates: Nothing to see here  https://review.openstack.org/31451011:22
derekhccamacho: testing a hunch ^ seeing if setting neutron_auth_type back to v3password helps (undoing https://review.openstack.org/#/c/312300/1)11:24
*** mgould is now known as mgould|afk11:26
*** thrash|g0ne is now known as thrash11:35
jistrderekh: yeah i saw the patch changing that value going to puppet-nova. We seem to be having a failure when establishing the connection though, so as long as IP and port are unaffected by the 'password' vs. 'v3password' change, it should at least establish the connection, regardless of what the rest of the URL looks like. But it's probably worth a try nevertheless.11:41
*** panda has quit IRC11:42
*** panda has joined #tripleo11:42
*** jaosorior has quit IRC11:42
*** jaosorior has joined #tripleo11:43
openstackgerritDougal Matthews proposed openstack/tripleo-common: WIP Testing and tweaks ontop of https://review.openstack.org/#/c/282366  https://review.openstack.org/31452011:45
d0ugalrbrady: https://review.openstack.org/#/c/314520/11:45
d0ugalrbrady: I'd just pull those changes into yours, I wasn't sure if I should add a new patchset or not - so take what you find useful from that.11:46
*** ccamacho is now known as ccamacho|lunch11:46
*** weshay has joined #tripleo11:48
rbradyd0ugal: wow.  thanks!11:49
*** lucasagomes is now known as lucas-hungry11:52
*** karthiks has quit IRC11:52
*** saneax is now known as saneax_AFK11:56
*** gfidente has joined #tripleo11:57
*** jtomasek_ has joined #tripleo11:58
*** pkovar has joined #tripleo12:00
derekhjistr: I've a feeling that maybe v3password choose the internalurl endpoint and for some reason the password plugin is using publicurl12:01
derekhjistr: but its just a hunch, we'll see what happens12:01
d0ugalrbrady: np, hopefully it helps.12:02
*** social has joined #tripleo12:03
*** karthiks has joined #tripleo12:04
*** jcoufal has joined #tripleo12:04
jistrderekh: yea that could be it perhaps. Also, the time when the change was made in puppet-nova is near the time the upgrade jobs started failing.12:05
derekhjistr: yup12:05
*** saneax_AFK is now known as saneax12:05
derekh<openstackgerrit> Derek Higgins proposed openstack/tripleo-heat-templates: Nothing to see here  https://review.openstack.org/31451012:05
derekh<derekh> ccamacho: testing a hunch ^ seeing if setting neutron_auth_type back to v3password helps (undoing https://review.openstack.org/#/c/312300/1)12:05
derekhgfidente: ^12:05
shardyjtomasek_: Can you by any chance provide me with a tripleo-ui screenshot, showing where we prompt for per-service configuration parameters?12:08
*** jtomasek_ has quit IRC12:08
shardyIn particular if you can show anything related to configuring Ceph or CephStorage roles that would be very helpful12:08
jtomasekshardy: sure12:09
* shardy really must get it running himself soon12:09
jtomasekshardy: actually, we're not doing it per service yet, we just list all the parameters in one modal12:10
*** ooolpbot has joined #tripleo12:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,Triaged]12:10
openstackgerritJiri Stransky proposed openstack/tripleo-docs: Upgrade documentation  https://review.openstack.org/30898512:10
shardyjtomasek: Ok, that's fine - just anywhere we surface any parameters related to Ceph derived from the templates will be fine12:11
jtomasekshardy: ok, give me a while, I'll get back to you12:11
shardyjtomasek: thanks!12:11
*** pradk has joined #tripleo12:16
*** dprince has joined #tripleo12:16
*** jayg|g0n3 is now known as jayg12:16
*** jcoufal has quit IRC12:21
openstackgerritRyan Brady proposed openstack/tripleo-common: Initial Integration of Mistral into TripleO  https://review.openstack.org/28236612:21
*** jpena|lunch is now known as jpena12:21
*** jcoufal has joined #tripleo12:22
rbradyd0ugal: I have pulled your change into the patch and added you as a co-author12:22
*** colonwq has quit IRC12:23
d0ugalrbrady: cool, thanks!12:23
*** dkehn has quit IRC12:23
*** dkehn has joined #tripleo12:24
*** dmacpher has joined #tripleo12:26
*** morazi has joined #tripleo12:27
*** aufi has quit IRC12:28
*** colonwq has joined #tripleo12:28
*** rlandy has joined #tripleo12:29
jtomasekshardy: https://goo.gl/photos/NFKeoQ2ZTxePetqeA12:29
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Add timestamps to postci log  https://review.openstack.org/31446812:30
*** MaxPC has joined #tripleo12:31
shardyjtomasek: thanks!12:31
*** ramishra has quit IRC12:34
rbradyd0ugal: re latest feedback on https://review.openstack.org/#/c/282366/ .  How strongly do you feel about the change of container to overcloud_name?12:34
openstackgerritMerged openstack/tripleo-common: StackAction parameter to distinguish top-level create vs. update  https://review.openstack.org/30639212:36
dprinced0ugal: hi, yeah I had an issue with your comment from patchset 22. To me "process_templates" has nothing to do with the overcloud_name I think12:36
dprinced0ugal: process_templates is just a utility function that could be used for a variety of things, validating stacks, creating stacks, updating stacks. etc12:37
*** dtantsur|bbl is now known as dtantsur12:37
dprinced0ugal: and as such I would prefer more generic names for the parameters12:37
*** liverpooler has quit IRC12:38
dprincerbrady: another question. Why are we passing in the swift base URL now again?12:38
dprincerbrady: Initially I was getting that from the swift endpoint directly12:39
*** sshnaidm has quit IRC12:39
d0ugalrbrady, dprince - sure, then it shouldn't set stack_name :) or it should have a more generic name than container12:39
d0ugalsince container clear maps to swift IMO12:39
dprinced0ugal: given the function requires the use of swift I would argue 'contianer' is the most right thing12:39
d0ugalalright, I don't really mind that much but it still seems wrong to me12:40
dprinced0ugal: we aren't abstracting this like the old API did12:40
dprinced0ugal: swift *is* our storage mechanism I think12:40
d0ugalYup, that's fair12:40
d0ugalbut that function also sets the stack_name, so it steps beyond just dealing with containers12:41
rbradydprince: your original code had this call to process_multiple_environments_and_files in an action which had access to the endpoint via the base action12:41
rbradydprince: when I moved this out to a utils method, I needed to pass in either the endpoint or the url, since I didn't use anything else from the endpoint I chose the url12:41
*** fultonj has joined #tripleo12:41
* shardy predicts folks will want git as an alternative to swift at some point12:41
dprinceshardy: git checkout, tarball, swift post. DONE!12:42
dprinceshardy: please... lets not mention that again :)12:42
shardyheh, fair enough ;)12:42
rbradydrpince: you'll notice I had to do the same with the auth_token as well because the utils method doesn't have direct access to the mistral context12:43
dprinceshardy: and if they did want to do it differently I would argue the code goes into python-heatclient anyways. So think about reviewing a git patch there...12:43
*** ccamacho|lunch is now known as ccamacho12:44
*** karthiks has quit IRC12:44
*** saneax is now known as saneax_AFK12:44
*** coolsvap has quit IRC12:47
dprincerbrady: http://paste.openstack.org/show/496567/12:48
dprincerbrady: we already use the Mistral ctx in the base class. So why not that ^^^12:48
*** coolsvap has joined #tripleo12:48
dprincerbrady: sorry if I seem nit-picky. My review is very much comparing my prototype code to what has evolved. I'm just trying to see things line up12:49
*** coolsvap has quit IRC12:50
*** coolsvap has joined #tripleo12:50
*** saneax_AFK is now known as saneax12:51
*** lucas-hungry is now known as lucasagomes12:51
EmilienMjistr: I'm reading your comment on https://review.openstack.org/#/c/314253/ - I'm going to move things to step5 but I still think step 6 is useless12:51
jtomasekdprince: what is the zaqar status right now, Is it already possible to listen to zaqar queue with current undercloud?12:51
rbradydprince: I'm not getting the value of adding that code for the swift url.  I can already get the url by calling self.get_object_client().url.  I still have to pass it to the utils method.12:51
EmilienMjistr: our CI is currently failing a lot of times, and upgade job is broken, because it takes too much time, we need to do something12:52
jtomasekdprince: I see you are using zaqar action here https://review.openstack.org/#/c/313953/2/workbooks/deployment.yaml12:52
dprincejtomasek: yes. I posted a fix of the weekend to fix an issue with subscribing to a queue though12:52
dprincejtomasek: https://review.openstack.org/#/c/313950/12:52
jistrEmilienM: yea i see that but i'm not sure removing step 6 is the way, at least not until we can say "yea we can do overcloud init via puppet even without step 6" /cc jaosorior12:52
dprincejtomasek: with that fix it all works for me :)12:52
jtomasekdprince: how can I apply that change to my undercloud?12:53
EmilienMjistr: at least we can try12:53
*** sshnaidm has joined #tripleo12:53
dprincejtomasek: edit /etc/zaqar/zaqar.conf directly and restart openstack-zaqar and openstack-zaqar@112:53
EmilienMjistr: do you have another solution to reduce CI jobs duration?12:54
jtomasekdprince: thanks!12:54
dprincerbrady: when you say utils which line are you referring to?12:54
rbradydprince line 99 in /utils/templates.py12:55
jistrEmilienM: image caching should help a ton i think.12:56
*** karthiks has joined #tripleo12:57
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: keystone: drop usage of step 6  https://review.openstack.org/31454312:57
*** mbound has joined #tripleo12:57
dprincerbrady: okay, I think it is fine then. I got confused and thought it was the action, I see your point. It is just the utils class12:58
dprincerbrady: going to try it out now12:58
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: deployment: drop step6  https://review.openstack.org/31425312:58
*** julim has joined #tripleo12:58
EmilienMjistr: I've looked at results, it helps but not "a ton". Dropping steps will help, every Puppet run on a controller is 7 minutes, that you can multiply by number of nodes and number of steps12:59
EmilienMslagle: can we land https://review.openstack.org/#/c/314242/ even if not passing upgrade job? It reduce (a little) the jobs)13:01
*** Goneri has joined #tripleo13:02
jaosoriorEmilienM: If we figure out that we can consistently deploy the keystone endpoints on step 5 (meaning that keystone will be up) Then I'm fine removing step 613:02
dprinceEmilienM: I would defer to jistr and marios for that one13:02
*** zoli|wfh is now known as zoli|lunch13:02
dprincejistr/marios: Was step 7 never used for pacemaker fencing (settling) or something?13:02
*** mbound has quit IRC13:02
EmilienMif I manage to make my patch passing CI consistently, I don't see the point of keeping step 613:02
jaosoriorhowever, the t-h-t cr that removes step 6, in it's current state, seems to break things13:02
jaosoriorEmilienM: I agree; having less steps is better13:03
EmilienMstep 6 + step 7 deletion can save us 25 min, + overcloud caching, we can save 35 min13:03
dprinceEmilienM: my concern is there is some case not covered by our CI related to multi-node HA upgrades or something13:03
*** mgould|afk is now known as mgould13:03
EmilienMjaosorior: yeah, see my patch in puppet-tripleo ^^^^13:03
jaosoriorEmilienM: I think step 7 is safer to remove, though13:03
jaosoriorEmilienM: Ah, just noticed it13:03
jistri think step 7 used to be for heat domain stuff, but got unnecessary after a few improvements to puppet-heat13:04
EmilienMgfidente: the point if removing steps is having less timeouts13:04
jistrstep 7 is probably safe to remove, step 6 not (yet)13:04
EmilienMgfidente: look http://tripleo.org/cistatus.html and all the red13:04
EmilienMgfidente: 7 steps is too much, I'm sure we can do everything in 5 or 4 steps later13:04
*** fpan has quit IRC13:05
EmilienMwe can do better at orchestration with resource ordering, tags, etc13:05
*** pkovar has quit IRC13:05
EmilienMdprince: for step7?13:05
jaosoriorEmilienM: Have you checked why the upgrades gate is failing on the CR that removes step 7? Is it always timeouts?13:05
dprinceEmilienM: I would like to modify the commit message a bit to avoid confusiong here. Perhaps referencing why step7 was added.13:05
gfidentethough ordering/tagging in the manifests makes harder to split per-service13:06
dprinceEmilienM: otherwise +2 on removing it from me13:06
dprinceEmilienM: good detective work on this13:06
EmilienMjaosorior: I almost spent all my day yesterday man13:06
EmilienMjaosorior: I figured some steps are running for almost nothing, and take ~25 min (step 6 and 7)13:06
EmilienMjaosorior: jobs are timeouting a lot for different reasons13:07
jaosoriorEmilienM: I haven't had much time to check, that's why I asked you. If it's failing the upgrades gate just because of timeouts... then screw it, lets merge that :P13:07
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: composable heat services  https://review.openstack.org/31357713:07
gfidenteso it's not that I don't want to remove unneeded steps, but I'd be careful about merging in same step stuff which isn't today in same step13:07
gfidentegiven the roles are stressing the ordering already13:07
EmilienMwe're not merging anything, right now I'm playing detective and trying to run 5 steps. If it works, maybe we can look at merging it. Right now I'm not sure yet13:08
EmilienMthough step7 can imho be removed safely13:08
*** akshai has joined #tripleo13:08
EmilienMbut we can't merge patches anymore until upgrade jobs are failing13:08
gfidenteyeah step 7 is fine to remove for sure13:09
gfidenteI'll go back looking at the upgrades job issue13:10
*** ooolpbot has joined #tripleo13:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,Triaged]13:10
openstackgerritBrad P. Crochet proposed openstack/python-tripleoclient: Remove the orchestration property from tripleoclient class  https://review.openstack.org/30855213:10
dprincejistr: so Step7 used to be Step6 before I incrented the Step counts for the composable services.13:11
dprincejistr: Step6 was added here: https://review.openstack.org/#/c/188415/13:11
EmilienMdprince: right, that's what I found13:12
EmilienMand I think fencing can be done in step5, no need of step6 for that.13:12
jistrdprince: yea i found that too.. but why did we add step7 here? https://github.com/openstack/tripleo-heat-templates/commit/9988bd25aa4bac1375ef4783d636c7adecedee9213:12
dprinceEmilienM: I would like to mention these details in the commit message I think13:12
EmilienMdprince: yeah I got it man13:13
dprincejistr: because it existed. And all I did was increment it13:13
EmilienMdprince: I'm just waiting for CI before updating it13:13
dprinceEmilienM: cool. I won't touch it then13:13
gfidentedprince, jistr I think that step got merged with one of the patches meant to do keystone users from puppet13:13
gfidentethen we reverted the change doing the users via puppet but not the one adding the step13:13
slagleEmilienM: imo, we can't land that w/o passing upgrades13:14
EmilienMdprince: oh, for step7, ok sorry13:14
jistrgfidente: ah right that could be it...13:14
jistrslagle: +113:14
gfidentekeystone endpoints actually, not keystone users13:14
slagleit's a template change removing a step, while we fully expect that to work on stack-update, we need to prove it :)13:14
mariosdprince: jistr: reading back13:14
slagleEmilienM: ^^13:14
EmilienMslagle: the thing is, I think upgrade job will come back with different fixes: reduce steps, caching, etc. And we need to manually evaluate every patch if it could break upgrade or not13:14
dprincejistr: I know why Step7 exists (I incremented the step counts). That was a simple thing and applied to all of the steps after ringbuilder so it doesn't concern me. My real question about this goes back to the addition of Step6 in https://review.openstack.org/#/c/188415/4 I think13:15
jistrdprince: so this is confusing and i had to read it a few times, but i think this change added one more step by mistake https://github.com/openstack/tripleo-heat-templates/commit/9988bd25aa4bac1375ef4783d636c7adecedee9213:16
slagleEmilienM: well we could make a case for landing it w/ failing upgrades if it's been manualy tested i suppose13:16
jistrdprince: what we should have done is renumber those steps, but not add additional ones, no?13:16
slagleEmilienM: as long as that is captured in the review. so someone deploys with tht w/o the patch, then stack-updates to tht with the patch13:16
dprincejistr: no, that was correct. The ringbuilder step wasn't in the "count" before13:16
dprincejistr: all I did was add it into the step numbering sequence so it could be de-composed with all the other services13:17
*** yamahata has joined #tripleo13:18
jistrdprince: yes but the patch made it to be step 3, and bumped the step numbers on the consecutive steps. So Step6 had indeed hiera('step') == 6 instead of hiera('step') == 5 as it was before. And that was all that needed to be done in that patch imo. Unless i'm missing something, addition of step 7 is superfluous.13:18
dprincejistr: I changed the manifests as well though13:20
*** pkovar has joined #tripleo13:20
dprincejistr: in the increment steps patch I changed it to:13:21
dprince$enable_fencing = str2bool(hiera('enable_fencing', false)) and hiera('step') >= 613:21
jistrdprince: yes, but also, hiera('step') == 5 was the maximum before, and after your patch hiera('step') == 6 was the maximum in the manifests13:21
dprincejistr: where it used to be 5. So it was all fine13:21
dprincejistr: the fencing step always used >=13:22
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: deployment: remove Step7  https://review.openstack.org/31424213:22
jistrdprince: so we went from Step6 = hiera5 to Step6 = hiera6, both in templates and in puppet manifests, which is correct, but still no need for step 7 imo.13:23
*** links has quit IRC13:23
dprincejistr: right. The rename didn't effect anything. But my real question then is back to this: https://review.openstack.org/#/c/188415/4/puppet/manifests/overcloud_controller_pacemaker.pp13:23
dprincejistr: that added the extra step6 at the time (which is now called step7)13:24
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: deployment: drop step6  https://review.openstack.org/31425313:25
EmilienMjistr: why don't we enable fencing by default in our CI btw?13:26
jistrdprince: hehe i still think we don't understand each other. Step6 is *still* called Step6, but it now also uses hiera value 6.13:26
mariosjistr: so the last step (i would have said it was 6, i don't recall 7) was for heat/keystone domain stuff... am looking at the review comments from jaosorior at https://review.openstack.org/#/c/314253 - have a call starting in 5 mins which I need to prepare for - will try and catchup later13:26
*** egafford has joined #tripleo13:26
jistrso: Step6 -- added because of fencing, might not be needed for fencing with EmilienM's patch, but still is used for keystone init13:27
*** lblanchard has joined #tripleo13:27
jistrStep7 -- never used, looks like it was added mistakenly during a refactoring of step numbers13:27
jistr(the step numbers should only be bumped on puppet side, to sync the Heat Step numbering with hiera step numbering, but no extra step addition was needed)13:28
*** mgould has quit IRC13:29
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Add loadbalancer profile for ha & non-ha  https://review.openstack.org/31302613:30
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Add loadbalancer profile for ha & non-ha  https://review.openstack.org/31302613:30
*** ramishra has joined #tripleo13:31
*** mgould has joined #tripleo13:31
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Deploy Loadbalancer as a composable role  https://review.openstack.org/31302913:32
*** ramishra has quit IRC13:35
dprincejistr: I disagree with your assessment that an extra step wasn't required for the step incrementing patch. Note that I removed ControllerRingbuilderPuppetConfig, and then added Step 7. The delta is 0.13:36
jistrdprince: well you removed the *config*, but not the *deployment*. The deployment was only renamed, so i think the delta is +1, as shown here: https://paste.fedoraproject.org/364649/87220146/13:37
* jistr double checks13:37
EmilienMslagle: thx fo the co-authoring on https://review.openstack.org/#/c/309408/ ! but no worries for next times, you can skip it13:38
*** sshnaidm has quit IRC13:39
dprincejistr: if that is the case then I think we list that as the root cause here and move on :)13:39
jistrdprince: yea for Step7 indeed -- can be safely removed, IMO. Step6 is still used though in puppet-tripleo for Keystone init.13:41
jistrEmilienM: ^13:41
dprincejistr: agree13:41
*** sshnaidm has joined #tripleo13:41
EmilienMjistr: and I dropped it13:41
dprincejistr: thanks for talking through this :)13:41
*** bvandenh has quit IRC13:42
jistrdprince: sure thing, thank you :)13:42
*** bvandenh has joined #tripleo13:42
dprinceEmilienM: so with regards to the removal of step6 you would be looking at https://review.openstack.org/#/c/188415/413:43
dprinceEmilienM: so if we can remove that too then just reference it. :)13:43
*** jobewan has joined #tripleo13:43
EmilienMdprince: did you see https://review.openstack.org/#/c/314253/ ?13:43
dprinceguys, I think I win the award for wasting the most CI time this year. Step 7!!13:43
EmilienMdprince: congrats man13:44
dprinceoh the public shame man!13:44
jistrEmilienM, dprince: and jaosorior's https://review.openstack.org/#/c/304125/ should be thought about too when removing step 6, but that might "just work" with what EmilienM did in https://review.openstack.org/31454313:44
dprinceEmilienM: cool, I will follow up on that13:44
slaglewe've got to come up with a $$$ value on this wasted time13:46
slaglehow much did the power cost? :)13:47
*** tzumainn has joined #tripleo13:47
gfidenteI bet money on step6 and maybe even step7 coming back as soon as we get more into the roles stuff13:48
openstackgerritMerged openstack/puppet-tripleo: Add tripleo::selinux  https://review.openstack.org/30940813:49
*** veteran has quit IRC13:49
slagleEmilienM: i had to give you and yanis co-author on that ^...i just basically copied it :)13:49
slagleplus i dont want anyone to think i wrote that much puppet13:50
EmilienMI think we stole it too lol13:50
EmilienMslagle: you can add "Puppet" in your Linkedin13:50
EmilienMI'll recommend you13:50
*** dsariel has quit IRC13:54
*** coolsvap has quit IRC13:54
*** ebalduf has joined #tripleo13:55
openstackgerritFlorian Fuchs proposed openstack/tripleo-ui: Adds a tarball upload option to plan creation  https://review.openstack.org/31458213:55
pabelangerderekh: thanks for the review. dprince: EmilienM: do you mind looking at https://review.openstack.org/#/c/312725/ that's the current patchset to support centos-7 jenkins slaves13:56
*** tosky has quit IRC13:57
*** yamahata has quit IRC13:57
*** yamahata has joined #tripleo13:57
openstackgerritFlorian Fuchs proposed openstack/tripleo-ui: Adds a tarball upload option to plan creation  https://review.openstack.org/31458213:57
EmilienMpabelanger: thing is, upgrade jobs are failing, and we decided to stop merging patches (except if upgrades were manually tested)13:58
EmilienMI would like to merge it, but I'm afraid we need to fix our upgrade jobs first13:59
shardyHey folks, meeting time14:00
dprincepabelanger: ack, I will look at it14:00
*** afazekas has quit IRC14:01
*** afazekas has joined #tripleo14:01
*** dsariel has joined #tripleo14:09
*** ooolpbot has joined #tripleo14:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,Triaged]14:10
*** julim has quit IRC14:11
*** zoli|lunch is now known as zoli|wfh14:11
*** julim has joined #tripleo14:12
*** oshvartz has quit IRC14:13
*** ppradhan has joined #tripleo14:13
*** rajinir has joined #tripleo14:20
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: composable heat services  https://review.openstack.org/31357714:22
*** jaosorior has quit IRC14:24
*** saneax is now known as saneax_AFK14:33
*** saneax_AFK is now known as saneax14:33
*** dc has joined #tripleo14:40
*** aufi has joined #tripleo14:41
*** jprovazn has quit IRC14:43
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-quickstart: Move image cache dir to a different location  https://review.openstack.org/31355314:43
openstackgerritEmilien Macchi proposed openstack/tripleo-puppet-elements: bump puppet-ceph to stable/hammer  https://review.openstack.org/31431114:44
*** ramishra_ has quit IRC14:49
openstackgerritRyan Brady proposed openstack/tripleo-common: Initial Integration of Mistral into TripleO  https://review.openstack.org/28236614:51
*** pcaruana has joined #tripleo14:53
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge.  https://review.openstack.org/31071314:57
*** dtrainor has joined #tripleo14:57
*** stendulker_ has joined #tripleo14:58
*** dsariel has quit IRC14:58
dprinceso who created this https://etherpad.openstack.org/p/tripleo-composable-roles-work14:59
EmilienMshardy: I think it's my fault15:00
dprinceall of that goes away I think :)15:00
EmilienMyour etherpad is much older15:00
dprinceshardy: I like my etherpad into the BP you created15:00
dprinceshardy: FWIW my style for spec light stuff is to link to etherpads in the LP specs15:00
*** masco has quit IRC15:01
dprinceshardy: keeping the 'implements: blueprint' stuff in the commit messages is fine too but I find etherpads help organize the sequencing, etc.15:01
*** milan has quit IRC15:04
*** tosky has joined #tripleo15:05
*** saneax is now known as saneax_AFK15:05
openstackgerritFlorian Fuchs proposed openstack/tripleo-ui: [WIP] Adds a tarball upload option to plan creation  https://review.openstack.org/31458215:06
openstackgerritFlorian Fuchs proposed openstack/tripleo-ui: [WIP] Adds a tarball upload option to plan creation  https://review.openstack.org/31458215:06
*** trozet has joined #tripleo15:07
*** ooolpbot has joined #tripleo15:10
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,Triaged]15:10
*** ooolpbot has quit IRC15:10
*** dsariel has joined #tripleo15:12
pabelangerdoh, missed the weekly meeting15:13
*** r-mibu has quit IRC15:14
*** r-mibu has joined #tripleo15:14
beaglesshardy, EmilienM: I'm cool with "workaround" for a bug tag as discussed. Should I just proceed with that?15:15
EmilienMcurrently in irc puppet meeting15:15
EmilienMbut yeah wfm15:15
beaglesEmilienM: thx15:15
*** Guest64153 has quit IRC15:16
dprinceEmilienM: I'm moving stuff from your etherpad into mine BTW.15:19
*** paramite is now known as paramite|afk15:19
dprinceEmilienM: seriously how did I not know about your etherpad man!?15:19
EmilienMdprince: you serious?15:19
dprinceEmilienM: no idea. Never seen it15:19
EmilienMdprince: I sent an email to tripleo ML after summit15:19
EmilienMdprince: FWIW, mine  has much more links & people connected.15:20
EmilienMbut feel free to move stuffs, as long people use it and it's useful.15:20
dprinceEmilienM: sorry, I totally missed that. Perhapd the URL was similar enough that I skipped it15:20
*** aufi has quit IRC15:22
*** kbyrne has quit IRC15:22
*** hewbrocca-afk is now known as hewbrocca15:24
dprinceEmilienM: for every new services that we would have deployed with Pacemaker, do not deploy it with pacemaker anymore because of the pacemaker-lite spec.15:24
EmilienMdprince: I disagree15:24
dprinceEmilienM: I'm removing that because incorrect I think. For now we do buth15:24
dprinceEmilienM: that was in the other etherpad15:24
dprinceEmilienM: I'm not copying it...15:24
*** rhallisey has quit IRC15:24
EmilienMI'm in a meeting...15:24
EmilienMbrb ~15 min15:24
*** rhallisey has joined #tripleo15:25
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: composable heat services  https://review.openstack.org/31357715:26
sshnaidmbnemec, what do you think? http://logs.openstack.org/68/314468/2/check-tripleo/gate-tripleo-ci-f22-nonha/af112b8/logs/postci.txt.gz15:29
bnemecsshnaidm: About what?15:34
sshnaidmbnemec, about adding timestamps to postci log to index it in logstash15:35
bnemecsshnaidm: Oh, yeah I'm all for that if it gets us the ability to logstash postci.15:35
*** kbyrne has joined #tripleo15:36
sshnaidmbnemec, great, so you can vote :) https://review.openstack.org/#/c/314468/15:36
*** milan has joined #tripleo15:39
*** rbrady has quit IRC15:39
*** rhallisey has quit IRC15:39
*** rbrady has joined #tripleo15:40
*** kbyrne has quit IRC15:40
*** rhallisey has joined #tripleo15:40
*** itamarl has quit IRC15:40
*** panda has quit IRC15:42
*** panda has joined #tripleo15:43
*** paramite|afk is now known as paramite15:44
*** mikelk has quit IRC15:45
dprinceEmilienM: no rush. But I don't see the email to openstack-dev about the composable services etherpad. I was going to reply with the new link...15:45
shardybeagles: sorry, also in mtg, +1 on workaround15:46
*** yamahata has quit IRC15:47
*** paramite has quit IRC15:48
dprinceEmilienM: no, I got it. Sorting issue15:48
*** dsariel has quit IRC15:48
EmilienMdprince: http://lists.openstack.org/pipermail/openstack-dev/2016-May/093566.html15:48
dprinceEmilienM: got it. replied. Really sorry I missed that one. Timing/travel :/15:51
dprinceEmilienM: anyways the etherpads are combined and I made some comments on the old with with respect to the pacemaker requirements15:51
EmilienMdprince: no worries I don't care.15:51
*** tesseract has quit IRC15:52
*** myoung has joined #tripleo15:54
*** sshnaidm has quit IRC15:57
*** tremble has quit IRC15:58
*** dprince has quit IRC15:58
*** cwolferh has joined #tripleo16:00
beaglesshardy: cool, thx16:01
*** penick has joined #tripleo16:04
EmilienMslagle: ha job looks faster, isn't? https://review.openstack.org/#/c/314543/16:06
EmilienM1h 56m 19s16:06
EmilienMoh nevermind, I found some jobs where is was 2h0016:07
*** eil397 has joined #tripleo16:08
derekhEmilienM: 1h 44 https://review.openstack.org/#/c/312397/16:09
EmilienMyeah... nevermind16:09
EmilienMthe THT patch I did will tell us16:09
*** ooolpbot has joined #tripleo16:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,Triaged]16:10
*** ramishra has joined #tripleo16:11
*** dprince has joined #tripleo16:14
*** bvandenh has quit IRC16:20
*** ifarkas has quit IRC16:28
EmilienMdprince: is https://review.openstack.org/#/c/314311/ fine for you now?16:31
*** electrofelix has joined #tripleo16:36
*** yamahata has joined #tripleo16:36
*** ohamada has quit IRC16:38
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: composable heat services  https://review.openstack.org/31357716:42
*** gfidente has quit IRC16:48
*** gfidente has joined #tripleo16:48
*** derekh has quit IRC16:50
*** sshnaidm has joined #tripleo16:53
*** stendulker_ has quit IRC16:53
*** tserong has quit IRC16:54
*** mgould has quit IRC16:55
*** eil397 has quit IRC16:55
*** eil397 has joined #tripleo16:56
*** mgould has joined #tripleo16:58
*** mgould is now known as mgould|afk16:59
*** tserong has joined #tripleo16:59
*** penick has quit IRC16:59
dprinceEmilienM: Just so I'm clear puppet-ceph master is targetted for jewel then?16:59
EmilienMdprince: yes17:00
jdobif i post a patch against heat and then add a depends-on to it from the tripleo repo, will tripleo's CI pick up that patch? gfidente and I were talking about it yesterday but we weren't 100% sure17:02
shardyjdob: yes, it should17:03
jdobawesome, thanks shardy17:03
shardyOr at lest, it did last time I tried it :)17:03
jdobshardy: and I can WIP the heat one and CI won't ignore it, right?17:03
gfidenteshardy jdob yeah this is useful for the get_addr issue testing17:04
shardyjdob: yup - the only gotcha is, when the heat patch actually merges, tripleo-ci will still use the pinned current-tripleo version17:04
jdobya, i'm about to vomit log statements all over heat, so i dont want it to merge17:04
jdobyet at least :)17:04
shardyjdob: ack, AFAIK it should work17:05
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: WIP: Force Nova to the Keystone internal endpoint  https://review.openstack.org/31468417:07
*** shivrao has joined #tripleo17:07
gfidentethere is no support in puppet-nova for service_auth yet ^^17:07
gfidentebut I think that might fix the problem we're seeing in upgrades17:07
gfidenteif it does, we can probably work an actual addition to the puppet module17:07
*** devvesa has quit IRC17:09
jdobgfidente: also, I'm gonna dork with the THT patch a bit too and mark that as WIP17:09
gfidentejdob sure17:09
*** shivrao_ has joined #tripleo17:10
jdobthe next step after this is to sacrifice a goat to see if I can rid the evil spirits from heat17:10
*** ooolpbot has joined #tripleo17:10
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,In progress] - Assigned to Giulio Fidente (gfidente)17:10
*** ooolpbot has quit IRC17:10
jdobwow, that bot really is a seagull17:11
*** tserong has quit IRC17:12
*** shivrao has quit IRC17:12
*** shivrao_ is now known as shivrao17:12
*** jpena is now known as jpena|off17:13
*** tserong has joined #tripleo17:13
openstackgerritJay Dobies proposed openstack/tripleo-heat-templates: [WIP] Pass MysqlVirtualIP via EndpointMap  https://review.openstack.org/30575117:15
*** athomas has quit IRC17:17
*** jistr has quit IRC17:21
*** MaxPC has quit IRC17:21
gfidentejdob the depends-on17:22
jdobstill working on the heat part17:22
jdobbut i wanted to kick that off17:22
gfidentetty tomorrow!17:22
jdobthough in hindsight, i probably should have waited17:22
jdobnight :)17:22
*** gfidente has quit IRC17:22
*** zoli|wfh is now known as zoli|gone17:23
*** zoli|gone is now known as zoli_gone-proxy17:25
openstackgerritBrad P. Crochet proposed openstack/puppet-tripleo: Add Heat profiles  https://review.openstack.org/31301917:25
dprinceshardy, slagle: this passes the upgrades job https://review.openstack.org/#/c/314510/1/puppet/hieradata/compute.yaml17:27
dprinceEmilienM: ^^^17:27
dprinceDoes that mean the puppet-keystone patch was perhaps the root cause of our issues?17:28
dprinceSorry, puppet-nove17:29
slagleshall we update the commit msg and merge it?17:30
*** penick has joined #tripleo17:30
dprinceslagle: I think so. Probably worth investigating a revert in puppet-nova as well.17:30
*** mcornea has quit IRC17:31
dprinceslagle: I will push a new patch and +2 it. No changes other than commit message and a comment maybe.17:31
shardyHas anyone verified that switching those settings manually in the nova conf breaks auth?17:31
shardy+1 on merging tho17:31
dprinceshardy: I've not. But we've not had upgrades passing for days man17:31
shardyI had some weird issues trying to use the generic password vs v3password plugin in heat recently17:31
*** TSCHAK_ has joined #tripleo17:32
shardydprince: Yeah, sure, I'm just thinking that should be a pretty quick confirmation, as those auth plugin settings generally either work or they don't17:32
shardyin particular the puppet stuff looks suspect because it mixes a generic (unversioned) password plugin with a v3 suffixed auth url17:32
* dprince finds bug reference17:33
*** sambetts is now known as sambetts|afk17:33
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,In progress] - Assigned to Giulio Fidente (gfidente)17:33
*** rooooook has quit IRC17:34
*** rooooook has joined #tripleo17:34
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: Set nova neutron auth back to 'v3password'.  https://review.openstack.org/31451017:35
dprinceslagle, shardy; I'd say we +A now17:36
*** dmacpher is now known as dmacpher-afk17:36
slagleWorking ...17:37
slagleWorking ...17:37
slaglethere it goes17:37
slaglegerrit is slow today17:38
slagledprince: i wonder if there is another bug in puppet somehow where it doesn't apply the config the first time, then on stack-update it does17:38
dprinceslagle: could be17:39
dprinceslagle: https://review.openstack.org/31469517:40
dprinceslagle: I'll ask Emilien and Colleen17:41
*** shardy has quit IRC17:41
EmilienMdprince: +217:44
*** lucasagomes is now known as lucas-afk17:49
openstackgerritMerged openstack/tripleo-heat-templates: Set nova neutron auth back to 'v3password'.  https://review.openstack.org/31451017:50
*** absubram has joined #tripleo17:50
*** bvandenh has joined #tripleo17:51
EmilienMdprince, slagle: wdyt if we run upgrade jobs in puppet CI beside non-ha? would it be too much?17:51
dprinceEmilienM: we could switch to just the upgrades job then?17:51
dprinceEmilienM: so long as we run a single job for each puppet patch I think it would be fine17:52
dprinceEmilienM: plus, now that you've gotten rid of a step who knows. Maybe it'll even run fast :)17:53
EmilienMdprince: ok i'm submitting a patch.17:53
dprincegotta watch that dprince, that guys always added un-needed code everywhere17:53
EmilienMdprince: upgrade job is HA, right?17:53
*** tosky has quit IRC17:53
EmilienMahahah no worries17:53
*** larstobi has joined #tripleo17:54
dprinceEmilienM: upgrades job is only single controller HA, but it does use pacemaker17:54
*** pkovar has quit IRC17:55
EmilienMoh ok17:55
*** lucas-afk is now known as lucasagomes17:55
EmilienMwell, let's do it17:55
openstackgerritJay Dobies proposed openstack/tripleo-heat-templates: [WIP] Pass MysqlVirtualIP via EndpointMap  https://review.openstack.org/30575117:55
* EmilienM preparing a patch17:55
dprincecrinkle: hi, a rather cryptic bug we seem to have traced back to that puppet-nova patch. I proposed a revert without spending too much time on it but perhaps you can double check it?17:56
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Add timestamps to postci log  https://review.openstack.org/31446817:56
*** lucasagomes is now known as lucas-dinner17:57
crinkledprince: very strange, i'll approve after i dig through the logs for 313650 for a bit17:58
dprincecrinkle: sure, np. We've just landed a fix in TripleO to avoid the issue so I think we aren't in a rush now17:59
EmilienMdprince: for puppet-pacemaker, I would make an exception and have both ha & upgrade jobs18:00
EmilienMthis module is very critical and does not have much contributions, so we can have 2 jobs.18:00
dprinceEmilienM: agree18:00
dprincegood exception to make18:00
dprinceEmilienM: lets loop derekh in on this too in case he has some resource concern I'm unaware of. I think it'll be fine though18:01
EmilienMsure, I'll mention it in commit message18:02
EmilienMdprince: https://review.openstack.org/31470818:06
*** rajinir has quit IRC18:06
*** mcornea has joined #tripleo18:08
*** rcernin has quit IRC18:13
*** florianf has quit IRC18:19
*** electrofelix has quit IRC18:19
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Composable role for RabbitMQ  https://review.openstack.org/31184918:19
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Deploy RabbitMQ as a composable role  https://review.openstack.org/31186318:20
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: Add loadbalancer profile for ha & non-ha  https://review.openstack.org/31302618:20
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Deploy Loadbalancer as a composable role  https://review.openstack.org/31302918:21
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: deployment: remove Step7  https://review.openstack.org/31424218:24
*** dtantsur is now known as dtantsur|afk18:36
*** mcornea has quit IRC18:38
*** penick has quit IRC18:41
*** penick has joined #tripleo18:41
*** rcernin has joined #tripleo18:42
*** pkovar has joined #tripleo18:43
EmilienMslagle, dprince: is there some WIP to have an upgrade job that test mitaka -> newton already?18:44
slagleEmilienM: the first step is to get the job times down18:45
*** bvandenh has quit IRC18:45
slaglewe're hoping the caching plus the hw ram/ssd's will buy us enough time so we can run such an upgrades job18:45
slagleas is, there is no reason to add one, because it would never finish within the gate timeout18:46
*** jcoufal has quit IRC18:47
EmilienMslagle: it makes sense18:49
dprincerbrady: hi, two very minor comments. https://review.openstack.org/#/c/28236618:50
*** olap has quit IRC18:50
dprincerbrady: I've just tested your patch so with those fixed I'm +218:51
dprincerbrady: want me to fix them?18:52
EmilienMslagle, dprince: another option for later (maybe one day) when we'll have sorted things out around CI & hw & etc: having already-deployed undercloud/overcloud in a pool of resources, and use them for upgrade jobs18:52
*** rajinir has joined #tripleo18:53
rbradydprince: wrt to the comment about heat and auth token, it's used in line 144, which calls into heat at line 148 and from looking at python-heatclient it seems it is used18:54
rbradydprince: I can fix the setup.cfg and add "keystone auth token for interacting with heat and swift".  would that suffice?18:57
dprincerbrady: yes. sorry the auth_token was is totally invalid18:58
*** fpan has joined #tripleo18:58
dprincerbrady: I see it now :)18:58
rbradydprince: okay, removed the line in setup.cfg19:01
openstackgerritRyan Brady proposed openstack/tripleo-common: Initial Integration of Mistral into TripleO  https://review.openstack.org/28236619:01
*** fragatina has quit IRC19:02
dprincerbrady: so the auth_token suggestion I had was actually right. What is going on is python-heatclient calls into Swift though. So the comment is fine, but it does actually eventually call Swift19:04
dprincerbrady: oh, and can we add Depends-On: Ibb98013d6ef283bd454f958bb036557d2b02997f19:04
dprincerbrady: that way the jenkins test should pass I think19:04
dprincerbrady: anyways, +2. Thanks man19:05
openstackgerritRyan Brady proposed openstack/tripleo-common: Initial Integration of Mistral into TripleO  https://review.openstack.org/28236619:06
rbradydprince: done.  thanks for all of the help19:07
dprincerbrady: cool, +2 again19:07
dprincerbrady: so I'd like to enable automatic cleanup of old executions19:08
dprincerbrady: Mistral already supports this, but its off by default19:08
openstackgerritMerged openstack/tripleo-quickstart: Move image cache dir to a different location  https://review.openstack.org/31355319:08
dprincerbrady: does 48 hours seem like a reasonable time to keep old executions?19:08
rbradydprince: I think that's a question best solved by asking an operator (or advocate).  48 hours seems reasonable to me, but I know I can turn that off if debugging something more long term19:09
*** fragatina has joined #tripleo19:09
dprincerbrady: sure, I'll throw up a patch and ask around I guess19:10
rbradydprince: maybe we add the feature and document a note on disabling in case of need19:10
dprincerbrady: having no cleanup seems bad though19:10
rbradydprince: agreed19:10
*** fpan has quit IRC19:12
*** pcaruana has quit IRC19:12
openstackgerritDan Prince proposed openstack/instack-undercloud: Enable Mistral execution cleanup  https://review.openstack.org/31473319:16
openstackgerritDimitri Savineau proposed openstack/tripleo-heat-templates: Remove duplicate neutron_mechanism_drivers  https://review.openstack.org/31473419:17
rbradydprince: do those settings show up in /etc/mistral/mistral.conf?19:19
dprincerbrady: yeah. one sec19:19
dprincerbrady: for a really short cleanup (2 hours)19:20
dprincerbrady: that would let you test it out if you want to tinker with it19:20
rbradydprince: thanks...adding that info to the notes for docs19:21
dprincerbrady: then restart mistral-engine19:21
dprincerbrady: cool19:21
slaglein which i find myself needing a yaml equivalent of jq19:22
*** MaxPC has joined #tripleo19:22
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: composable neutron ml2 and ovs agent  https://review.openstack.org/30851419:23
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: composable neutron dhcp service  https://review.openstack.org/30338619:23
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: composable neutron metadata service  https://review.openstack.org/30361819:23
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: composable neutron l3 service  https://review.openstack.org/30356219:23
dprinceslagle: maybe not what you are asking for but the yaql stuff is pretty nice19:23
slagleis there a cli?19:24
slaglewhat's the package name?19:24
dprinceI guess it loads json files though19:24
slaglehmm, can't find it on f2219:24
dprinceslagle: install python2-yaql on your undercloud (Centos)19:25
*** julim has quit IRC19:26
dprinceslagle: sadness it only loads json ATM19:26
slagleyea i see19:27
*** paramite has joined #tripleo19:27
slaglei want to be able to do something like "jq .parameters overcloud.yaml" and get just the parameters from overcloud.yaml in yaml format19:29
dprinceslagle: heat stack-validate?19:29
slagleno, for any yaml19:29
dprinceyeah, I get you19:29
dprinceslagle: just create a simple python class man19:30
*** paramite has quit IRC19:40
*** panda has quit IRC19:41
rbradydtrainor: what do you think of the 48 hour lifespan of old mistral executions?  Does that sound reasonable enough from an operators perspective?  should the time be shorter or longer?19:42
dtrainori think it sounds rather long19:42
dtrainoris this a hard default of some type?19:43
*** panda has joined #tripleo19:43
bnemecIt sounds short to me, coming from a customer who just left their overcloud in a bad state for weeks before getting back to it.19:44
rbradydtrainor: it's just a config setting.  it can be changed in a file and a service restarted to take effect.19:44
dtrainoras i understand executions, it doesn't sound like any of them should take that long19:44
rbradybnemec: thanks, this is valuable feedback to get some live operations info19:44
dtrainorUnless it's in a Waiting or Idle state, no?19:45
rbradydtrainor: it's information about past executions....history19:45
rbradydtrainor: e.g. you run validations that fail and you want to look at the output gathered19:45
dtrainoroh, that was not clear.  if that's the case then yes I'd agree with bnemec, I'd want as much of that as reasonably possible.19:45
dtrainori think 7d would be a reasonable but aggressive default19:46
openstackgerritEthan Gafford proposed openstack/tripleo-heat-templates: Pass MysqlVirtualIP via EndpointMap  https://review.openstack.org/30575119:49
openstackgerritEthan Gafford proposed openstack/tripleo-heat-templates: Trove Integration  https://review.openstack.org/23324019:49
dtrainorWould that be do-able?19:50
dprincedtrainor: a deployment execution may take hours19:51
dprincedtrainor: so you prefer a week?19:51
*** rcernin has quit IRC19:51
dtrainordprince, i understand, but rbrady was talking about historical data on completed tasks if I understand correctly19:52
dtrainorI'd want the historical data around for a pretty long time19:52
dprincedtrainor: https://review.openstack.org/#/c/314733/19:53
bnemecWe're not talking about an enormous amount of data here, are we?19:53
dprincebnemec: it all depends on what the workflows "publish"19:53
bnemecIn some cases it should just be a boolean that says validation passed/failed.19:53
dtrainorthat's why i cautioned with the word "reasonable", I'm not sure how much data there would be19:53
dprincebnemec: if they publish lots of data it could add up I think19:53
dprinceI expect there will be so many executions that at some point it'll be hard to keep track of them. Thus I'd like to see us clean things up19:54
dtrainorit would be a lot of records, sure19:55
dprincein my case, if a workflow isn't running after a few minutes I never check it again19:55
dprincein most of my cases so far that is19:55
dtrainorand that's fine, but i'd also say that's because you're dprince19:55
dtrainorothers not as familiar with this might need more information19:56
dprincelets put it this way. If we have long lived data it needs to come out of executions and quickly move into something else (swift or something)19:56
dprincethat is my take anyway19:56
dprincedtrainor: and yes, I have been creating lots of ad-hoc executions19:57
dtrainori agree in some capacity, but i wouldn't want to switch tools to view that "old old" data19:57
dtrainorthat's where the happy medium comes in to play i suppose19:57
dprincedtrainor: comment on that patch with your feedback. We can also start an email thread if that works19:57
dtrainorwill do.  thanks for running it past me rbrady19:57
*** rcernin has joined #tripleo19:58
jristhey rbrady on this patch https://review.openstack.org/#/c/282366/ why is it saying jenkins -1 even though all the gates passed?20:00
rbradyjrist: because jenkins is a greedy bastard who stopped accepting my bribes to +1 patches20:01
dprincejrist: the requirements issue20:01
dprincejrist: seems it didn't pickup the Depends-On then?20:01
rbradyjrist: what dprince said20:01
jristI see20:02
*** dprince has quit IRC20:05
*** absubram has quit IRC20:06
*** akshai has quit IRC20:08
*** Goneri has quit IRC20:12
rbradyjrist: did you see https://review.openstack.org/#/c/314638/ ?20:26
*** fragatina has quit IRC20:27
*** ebalduf has quit IRC20:27
*** dc has quit IRC20:29
*** absubram has joined #tripleo20:33
bnemecslagle: Regarding parameter editing in YAML, I've got two pieces of code that together would probably do what you need:20:39
bnemecLine 54 of https://review.openstack.org/#/c/253638/7/tripleo_heat_templates/environment_generator.py20:39
bnemecand https://github.com/cybertron/tripleo-scripts/blob/master/net_processing.py#L17620:39
*** ccamacho has quit IRC20:42
slaglebnemec: cool, my example however was arbitrary and isn't what i was actually doing :)20:44
slaglei literally want a "yq" interface that works for anything. there appear to be some random ones on github20:45
slagleb/c ya know, if i'm going to spend my days coding in yaml...20:46
slagle(finish however you like i guess)20:46
jristYet Another YAML Implementation20:46
* bnemec dies a little inside20:47
*** apetrich has quit IRC20:47
jristbnemec: :)20:48
*** lucas-dinner has quit IRC20:51
*** ccamacho has joined #tripleo20:54
*** lucasagomes has joined #tripleo20:58
openstackgerritPradeep Kilambi proposed openstack/puppet-tripleo: loadbalancer: add Aodh API support  https://review.openstack.org/28812221:05
*** ebalduf has joined #tripleo21:16
*** lblanchard has quit IRC21:16
*** rhallisey has quit IRC21:17
*** jayg is now known as jayg|g0n321:22
*** hjensas has quit IRC21:23
*** yamahata has quit IRC21:23
*** MaxPC has quit IRC21:25
*** egafford has quit IRC21:26
*** hjensas has joined #tripleo21:27
*** yamahata has joined #tripleo21:29
*** mbound has joined #tripleo21:30
openstackgerritPradeep Kilambi proposed openstack/python-tripleoclient: Aodh integration to tripleoclient  https://review.openstack.org/31478421:41
*** milan has quit IRC21:44
*** milan has joined #tripleo21:44
*** pkovar has quit IRC21:49
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: keystone: drop usage of step 6  https://review.openstack.org/31454321:49
openstackgerritPradeep Kilambi proposed openstack/python-tripleoclient: Aodh integration to tripleoclient  https://review.openstack.org/31478421:53
*** fultonj has quit IRC21:56
EmilienMHA jobs randomly fail because of sahara db sync that runs before galera ready21:56
*** fragatina has joined #tripleo21:57
*** rbrady has quit IRC21:59
*** ccamacho has quit IRC22:06
*** ccamacho has joined #tripleo22:09
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Ensure db-sync Execs are run when Galera is ready  https://review.openstack.org/31479222:10
EmilienMslagle: ^ and https://review.openstack.org/#/c/314791/ will maybe help us too, so we can run dbsync in a previous step22:11
EmilienMslagle: thanks to resource tagging22:11
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: deployment: drop step6  https://review.openstack.org/31425322:12
EmilienMall upgade jobs are failing, and it seems https://review.openstack.org/#/c/282366/ should help22:15
* EmilienM will continue later /me afk22:17
*** morazi has quit IRC22:43
*** Goneri has joined #tripleo23:03
*** jobewan has quit IRC23:07
*** saneax_AFK is now known as saneax23:20
*** d0ugal has quit IRC23:21
*** absubram has quit IRC23:26
*** rcernin has quit IRC23:28
*** Goneri has quit IRC23:30
*** d0ugal has joined #tripleo23:37
*** trozet has quit IRC23:38
*** rooooook has quit IRC23:39
*** rooooook has joined #tripleo23:43
*** rooooook has joined #tripleo23:44
*** crinkle has quit IRC23:55
*** chlong has joined #tripleo23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!