14:00:04 <EmilienM> #startmeeting tripleo
14:00:05 <openstack> Meeting started Tue Aug 23 14:00:04 2016 UTC and is due to finish in 60 minutes.  The chair is EmilienM. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:06 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:09 <openstack> The meeting name has been set to 'tripleo'
14:00:24 <EmilienM> #topic rollcall
14:00:27 <EmilienM> hello folks
14:00:32 <saneax> \o
14:00:33 <shadower> heya
14:00:36 <sshnaidm> o/
14:00:37 <mandre> hello
14:00:40 <dprince> hi
14:00:43 <chem> hi
14:00:44 <rhallisey> hi
14:00:48 <beagles> o/
14:00:59 <weshay> 0/
14:01:00 <akrivoka> hiya \o
14:01:03 <Jokke_> o/
14:01:19 <EmilienM> #topic agenda
14:01:28 <EmilienM> * review past actions items
14:01:32 <[1]cdearborn> o/
14:01:34 <jrist> o/
14:01:41 <EmilienM> * one off agenda items
14:01:43 <EmilienM> * bugs
14:01:45 <EmilienM> * Projects releases or stable backports
14:01:47 <EmilienM> * CI
14:01:49 <EmilienM> * Specs
14:01:51 <EmilienM> * open discussion
14:02:00 <thrash> o/
14:02:05 <ccamacho> o/
14:02:05 <EmilienM> #topic past action items
14:02:16 <florianf> o/
14:02:17 <EmilienM> 1) tripleo team to review https://review.openstack.org/#/c/298732/ and https://review.openstack.org/#/c/342261/
14:02:38 <EmilienM> one is merged the other is WIP
14:02:59 <EmilienM> looks like d0ugal needs more time
14:03:13 <hrybacki> o/
14:03:22 <EmilienM> 2) EmilienM to check & release stable branch this week if needed -> needs to be postponed, I haven't checked yet
14:03:35 <karthiks> o/
14:03:40 <radeks> o/
14:03:56 <rbrady> o/
14:03:58 <EmilienM> 3) EmilienM to propose periodic scenarios CI jobs testing on ML -> done. Discussion ongoing on ML and Gerrit, please review https://review.openstack.org/#/c/356675/
14:04:12 <adarazs> o/
14:04:25 <EmilienM> any question about past actions items?
14:04:47 <EmilienM> #topic one off agenda items
14:04:54 <EmilienM> mandre: go ahead
14:05:00 <tbarron|gone> hi
14:05:05 <coolsvap_> o/
14:05:05 <EmilienM> #link https://etherpad.openstack.org/p/tripleo-meeting-items
14:05:08 <mandre> so, I'd like we have a look at https://review.openstack.org/#/c/313632/
14:05:25 <mandre> it's a patch to bootstrap ansible framework on the undercloud
14:05:29 <jrist> we really need it for the UI
14:05:32 <mandre> it's used for validations
14:05:35 <jrist> for 10
14:05:40 <jrist> sorry I mean newton
14:05:40 <jrist> :)
14:05:43 <mandre> a couple of things came up during the review
14:05:45 <EmilienM> jrist: better ;-)
14:06:01 <mandre> 1. can we use ansible API directly?
14:06:16 <EmilienM> it seems like a disagreement between dprince and UI/API folks?
14:06:23 <jtomasek> o/
14:06:24 <d0ugal> Hey, sorry I am late :)
14:06:30 <mandre> 2. do we want to make this ansible framework more generic and use it for more than just validations
14:07:01 <mandre> the main blocking point, I think, is do we store the private SSH key on the undercloud
14:07:19 <EmilienM> wasn't it something defined in a spec?
14:07:26 <dprince> my concerns mostly rely about how we manage the private ssh key. I'm not a fan of storing such keys long term in the undercloud.
14:07:31 <EmilienM> I'm surprised to see this design conversation out of a spec
14:07:59 <dprince> if we were to use #1 we could avoid storing the key locally
14:08:32 <dprince> if we use #2 I think we need to take more care about storing the private key, writing it out, and possibly cleaning it up to accomidate the CLI version of Ansible
14:08:33 <mandre> we do have a spec, https://review.openstack.org/#/c/255792/
14:09:11 <jrist> dprince: will this all change if Tower is Open Source?
14:09:38 <dprince> jrist: perhaps, but we are essentially implementing a "poor mans tower" with this I think
14:09:48 <jrist> but we have no choice :)
14:09:48 <mandre> however, the spec doesn't cover in details how to boostrap ansible on the undercloud
14:10:23 <EmilienM> +1 with dprince
14:10:30 <EmilienM> mandre: that's a problem imho
14:10:39 <EmilienM> the spec should cover that thing
14:10:45 <jrist> specs are hard some times
14:10:49 <jrist> once you get going you discover new issues
14:10:54 <jrist> they are not perfect
14:11:01 <dprince> EmilienM: the spec doesn't get into this detail. ANd I think that is fine. It is okay to catch some things in review
14:11:14 <jrist> the problem is the timing, dprince
14:11:16 <EmilienM> it sounds like we need to amend the spec or at least continue the design discussions on ML or Gerrit
14:11:23 <dprince> I think the frustration is there has been several attempts at this and it is tricky
14:11:29 <jrist> we need a decision on a resolution today
14:11:54 <EmilienM> jrist: we can't fix all today ;-) we have multiple other topics that also need to be fixed today
14:12:01 <jrist> I didn't say fix it today
14:12:10 <jrist> just a decision on the direction
14:12:13 <EmilienM> I would rather try to find a concensus on the ML asap
14:12:17 <dprince> My thought is that Ansible is part of the problem here in that the lack of a "Stable" library/API is forcing us to store the SSH key. I'd much rather just see us use Ansible properly which IMO is this: http://docs.ansible.com/ansible/developing_api.html
14:13:05 <dprince> mandre: has pointed out we'd have to work around some Mistral action issues with that approach, but it gets us into a better (less statefull ssh key) place I think
14:13:06 <shadower> dprince: that API requires an SSH key as well
14:13:20 <shadower> dprince: or rather, Ansible depends on a passwordless SSH access to nodes
14:13:24 <dprince> shadower: yes, but it doesn't have to be written to disk in the same manner I think, right?
14:13:25 <mandre> dprince: I don't see how using the ansible python API would avoid persisting the key, maybe we can develop after the meeting
14:13:35 <shadower> dprince: it does
14:13:42 <shadower> dprince: as it currently stands at least
14:14:24 <dprince> mandre: you aren't blocked I think with the current approach
14:14:51 <dprince> mandre: just avoid reading the key from the undercloud. I've said this same thing in past reviews... I'm not a fan of that. Store it elsewhere and write it out as needed
14:14:54 <shadower> dprince: I've written an API that the UI was going to use before we decided to go with Mistral. It used the python API and its SSH key requirements were identical
14:15:20 <dprince> shadower: same comment then, all of my disklike is due to how we manage this private ssh key
14:15:52 <dprince> I think this isn't good in general, but it would conflict with future features like an HA undercloud
14:17:04 <mandre> dprince: maybe we can manage the private key outside of the undercloud in a next iteration?
14:17:46 <dprince> mandre: we could, yes. But I think it should be possible to do just as well now no?
14:18:10 <mandre> dprince: time constraints... M3 is approaching fast
14:18:14 <shadower> dprince: mind the time and review queues of the core devs.
14:18:35 <dprince> mandre, shadower: sorry if this seems harsh but I really feel like me concern here is nothing new. I'm not blocking in anyways. Just a -1 on storing the key in this manner
14:19:14 <mandre> right, this is the reason I want to involve more cores in the discussion
14:19:25 <dprince> mandre, shadower I'm willing to help brainstorm or implement this FWIW
14:20:53 <EmilienM> ok, 15 min on this item, can we follow-up on ML maybe?
14:21:03 <mandre> dprince: would it make it for newton, though?
14:21:27 <EmilienM> we might need FFE
14:21:58 <EmilienM> but I'm not the PTL, we need to check with shardy if he would agree
14:22:14 <shadower> when is he going to  be back?
14:22:29 <EmilienM> end of week probably
14:22:39 <jrist> ;(
14:22:43 <dprince> mandre: maybe, maybe not. Sometimes holding until it is right is a good thing
14:22:49 <EmilienM> I suggest you to continue this discussion on IRC/ML using public channels
14:23:11 <EmilienM> we need to move ahead to the next item
14:23:31 <EmilienM> mandre: please continue on ML
14:23:37 <EmilienM> I'm sure we can find concensus this week
14:23:42 <mandre> will do
14:24:26 <EmilienM> #action mandre to continue ansible/sshkey discussion on ML / Gerrit and find concensus with dprince
14:24:34 <EmilienM> chem: next item
14:24:40 <chem> hi, just wanted to give a little up on those M/N upgrade reviews.  There are very shorts.
14:25:02 <EmilienM> w00t, nice work here
14:25:09 <chem> that's basically all :)
14:25:27 <EmilienM> #action tripleo team to review upgrade patches: https://review.openstack.org/358022 , https://review.openstack.org/358006 , https://review.openstack.org/358005 , https://review.openstack.org/357022 , https://review.openstack.org/354713 , https://review.openstack.org/357750
14:25:42 <EmilienM> chem: thanks
14:25:50 <EmilienM> skramaja: your turn
14:26:06 <skramaja> thanks.. we have made a short recording on SR-IOV
14:26:07 <skramaja> https://bluejeans.com/s/63pWQ/
14:26:13 <EmilienM> #link SR-IOV demo https://bluejeans.com/s/63pWQ/
14:26:25 <skramaja> to demonstrate the ping between 2 VMS with SR-IOV nics..
14:26:26 <EmilienM> skramaja: thanks for sharing
14:26:30 <skramaja> comments are welcome..
14:26:31 <skramaja> and
14:26:38 <EmilienM> skramaja: don't hesitate to share it on the ML also
14:26:42 <skramaja> we have closed all the comments on specs. PTAL
14:26:46 <skramaja> sure..
14:26:52 <EmilienM> skramaja: ok so spec is ready for review?
14:26:58 <skramaja> yes.
14:27:18 <EmilienM> #action tripleo team to review sr-iov spec: https://review.openstack.org/#/c/313872/
14:27:24 <EmilienM> #link sr-iov spec https://review.openstack.org/#/c/313872/
14:27:40 <skramaja> thanks EmilienM thats all from my side..
14:27:50 <EmilienM> Jokke_: you're next
14:28:02 <Jokke_> just quick one
14:28:14 <Jokke_> Manilla Ceph FS Native integration
14:28:27 <Jokke_> Should be ready for reviews
14:28:38 <EmilienM> do you have a gerrit topic handy?
14:28:42 <Jokke_> http://lists.openstack.org/pipermail/openstack-dev/2016-August/101968.html https://review.openstack.org/#/c/354047/ https://review.openstack.org/#/c/358525/
14:29:15 <Jokke_> I do realize that we're on the last squeeze so I took bit advance and sent request for considering it as FFE candidate if it does not make it
14:29:28 <EmilienM> we'll have to check with shardy this request
14:29:35 <Jokke_> that's it from me, thanks!
14:29:38 <EmilienM> cool
14:29:55 <EmilienM> akrivoka: hey
14:30:18 <EmilienM> it sounds like it's about https://review.openstack.org/#/c/349532/
14:30:39 <EmilienM> #action tripleo team to review https://review.openstack.org/#/c/349532/ "Add default plan creation"
14:31:07 <akrivoka> EmilienM: yes, I just wanted to draw attention to that, we need it for better UI exp
14:31:10 <akrivoka> EmilienM: thanks
14:31:19 <EmilienM> oki
14:31:24 <EmilienM> any more items for today?
14:32:10 <EmilienM> ok let's go ahead in the agenda
14:32:14 <EmilienM> #topic bugs
14:32:40 <EmilienM> #link https://bugs.launchpad.net/tripleo/+milestone/newton-3
14:33:22 <EmilienM> so about $topic, I sent an email about launchpad, please subscribe to emails notifications
14:33:34 <EmilienM> also, as a reminder, please do self triage when you report a bug
14:34:02 <EmilienM> it seems like we have a bunch of bugs "in progress" -> make sure they are updated if you're assigned
14:34:31 <EmilienM> do we have outstanding bug we need to talk about this week?
14:34:42 <EmilienM> (/me comes back from PTO, I'm still catching up)
14:35:41 <EmilienM> slagle: I see 2 critical bugs in progress
14:35:45 <EmilienM> https://bugs.launchpad.net/tripleo/+bug/1612622
14:35:45 <openstack> Launchpad bug 1612622 in tripleo "CI: Exception registering nodes: Node 59ecc491-41b0-4950-8d00-ddce532ba0f4 is locked by host localhost.localdomain, please retry after the current operation is completed" [Critical,In progress] - Assigned to James Slagle (james-slagle)
14:35:48 <EmilienM> and https://bugs.launchpad.net/ironic/+bug/1613331
14:35:48 <openstack> Launchpad bug 1613331 in tripleo "ironic using public endpoint to configure ipxe template during deployment" [Critical,In progress] - Assigned to James Slagle (james-slagle)
14:36:09 <EmilienM> I think we can close them
14:36:47 <EmilienM> sounds like huge interest in bugs :-)
14:37:03 <sshnaidm> EmilienM, agree, it seems to be solved
14:37:30 <slagle> yes, the ironic one is fixed
14:37:41 <EmilienM> ok, let's close them
14:37:48 <EmilienM> anything else critical?
14:38:30 <EmilienM> ok  next topic
14:38:33 <EmilienM> #topic Projects releases or stable backports
14:38:40 <shadower> tripleo-validations has not seen a release yet (it's new this cycle) and it could use one to help downstream packaging
14:38:53 <EmilienM> #action EmilienM to check with shardy and the team if we want a new Mitaka release
14:38:55 <shadower> happy to help but I'm not sure how
14:39:12 <EmilienM> shadower: ok, I can help on this thing
14:39:26 <shadower> EmilienM: thanks
14:39:54 <EmilienM> shadower: https://review.openstack.org/#/c/340350/
14:39:58 <EmilienM> it's an example of tripleo release
14:40:15 <EmilienM> feel free to submit a patch and add shardy & me as reviewers.
14:40:28 <shadower> EmilienM: will do, thanks!
14:40:44 <EmilienM> we're currently in R-6
14:40:52 <EmilienM> which is the "Final release for non-client libraries" time
14:41:19 <EmilienM> next week will be R-5, newton-3 milestone, feature freeze
14:41:39 <EmilienM> and as a reminder, our final release will be during Oct 17-21 week
14:42:10 <EmilienM> I'll send a reminder on the ML about all dates we need to know.
14:42:23 <EmilienM> #action EmilienM to send email on ML about TripleO release schedule
14:42:45 <EmilienM> #link Ocata release proposal https://review.openstack.org/357214
14:42:57 <EmilienM> if you're interested in release for Ocata, here's the start of proposal ^
14:43:18 <EmilienM> any question about releases?
14:43:45 <EmilienM> #topic CI
14:44:09 <Jokke_> EmilienM: quick one ... are we trailing in Ocata as well?
14:44:19 <Jokke_> release related ^^
14:44:20 <EmilienM> sshnaidm, dprince, slagle, derekh: could you guys give an update about CI status please? (I was afk the last 4 days)
14:44:27 <EmilienM> Jokke_: yes
14:44:31 <Jokke_> ty
14:44:33 <derekh> Somebody else might want to take this, I'm hands off on CI these days (moving to a slightly different area)
14:44:35 <EmilienM> Jokke_: tailing is our release model
14:44:56 <sshnaidm> we had a promote yesterday for repo
14:45:03 <sshnaidm> (but not for images)
14:45:19 <dprince> EmilienM: there was an outage last week due to an infra change
14:45:32 <dprince> with regards to the nodepool credentials
14:45:55 <dprince> we reverted the nodepool credentials back to the previous value on friday and this fixed the OVB cloud
14:46:01 <derekh> sshnaidm: not for images?
14:46:16 <dprince> so, 24 hours or so of the outage on late Thursday/Friday was due to this issue
14:46:18 <sshnaidm> derekh, out of space on mirror host, images weren't saved
14:46:27 <EmilienM> dprince: ok, thanks
14:46:27 <derekh> sshnaidm: ahh ok
14:46:32 <EmilienM> shadower: :(
14:46:37 <EmilienM> oops, sshnaidm ^
14:46:37 <derekh> sshnaidm: https://review.openstack.org/#/c/348806/
14:46:51 <dprince> I haven't looked into the nonha job failures today but it seems to be acting up recenlty
14:47:12 <sshnaidm> dprince, slagle https://review.openstack.org/#/c/348806/
14:47:21 <jpeeler> dprince: can second that fwiw
14:47:24 <EmilienM> derekh: +A
14:47:35 <sshnaidm> I don't see any critical problem for now in CI except of long job duration
14:47:43 <EmilienM> ok
14:47:47 <EmilienM> anything else about CI?
14:48:59 <EmilienM> #topic Specs
14:49:06 <sshnaidm> just FYI, POC job of using tripleo-quickstart instead of instack works good
14:49:07 <EmilienM> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open
14:49:20 <EmilienM> sshnaidm: link?
14:50:22 <sshnaidm> #link https://review.openstack.org/#/c/358919/
14:50:50 <sshnaidm> just POC, an additional work is required
14:51:02 <EmilienM> cool
14:51:14 <EmilienM> anything about specs except what we said in off items?
14:51:41 <jpich> Have a look at the bug tagging policy, so we can merge the first iteration soon, maybe? :-)
14:51:45 <jpich> https://review.openstack.org/#/c/352852/
14:51:46 <EmilienM> it would be great to have a look at https://review.openstack.org/#/c/352852/
14:51:50 <EmilienM> jpich: in same time :)
14:51:57 <jpich> :D
14:52:15 <EmilienM> #action tripleo team to review bug tagging policy https://review.openstack.org/#/c/352852/
14:52:26 <jrist> thanks jpich
14:52:38 <EmilienM> antyhing else about specs?
14:53:02 <ahana__> Hello
14:53:14 <EmilienM> #topic open discussion
14:53:21 <EmilienM> sounds like we have 7 min for open discussion
14:53:42 <EmilienM> if you have any question or comment, please go ahead, it's the right time
14:54:08 <EmilienM> any volunteer to drive the next tripleo deep dive?
14:54:18 <EmilienM> I don't see anyone for Thursday https://etherpad.openstack.org/p/tripleo-deep-dive-topics
14:54:40 <ccamacho> Hey re OOM issues, in my case adding swap files everywhere solved the issue for me just FYI
14:55:04 <EmilienM> ccamacho: is there a launchpad # for this problem?
14:55:30 <ahana__> Hi is this the right forum to ask about neutron flavor implementation?
14:55:37 <ccamacho> mmm not really because in CI we are using swap
14:55:53 <ccamacho> but locally my deployments were quite fragile
14:56:04 <EmilienM> ahana__: not really, go on #openstack-neutron
14:56:20 <shadower> btw cores: a lot of the tripleo-validations patches have a +2 and are ready for merging: https://review.openstack.org/#/q/project:openstack/tripleo-validations+status:open
14:56:38 <ahana__> ok thanks EmilienM
14:56:44 <ccamacho> and adding some swap solved the issues (Its not a problem at this time) just something that we might consider
14:56:49 <shadower> I emaild the ML but haven't seen people looking much
14:57:18 <EmilienM> #action tripleo team to help in reviewing validation's work https://review.openstack.org/#/q/project:openstack/tripleo-validations+status:open
14:57:32 * shadower is happy to answer any questions
14:57:39 <EmilienM> shadower: thanks
14:57:53 <EmilienM> anything else for today? I'll close the meeting in 30s otherwise
14:58:35 <EmilienM> thanks everyone
14:58:42 <EmilienM> have a great week!
14:58:45 <EmilienM> #endmeeting