14:00:22 <EmilienM> #startmeeting tripleo 14:00:22 <openstack> Meeting started Tue May 16 14:00:22 2017 UTC and is due to finish in 60 minutes. The chair is EmilienM. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:26 <EmilienM> #topic agenda 14:00:27 <openstack> The meeting name has been set to 'tripleo' 14:00:32 <EmilienM> * review past action items 14:00:34 <EmilienM> * one off agenda items 14:00:36 <EmilienM> * bugs 14:00:38 <EmilienM> * Projects releases or stable backports 14:00:40 <EmilienM> * CI 14:00:42 <EmilienM> * Specs 14:00:44 <EmilienM> * open discussion 14:00:46 <EmilienM> Anyone can use the #link, #action and #info commands, not just the moderatorǃ 14:00:48 <EmilienM> Hi everyone! who is around today? 14:00:54 <jpich> o/ 14:01:04 <ccamacho> Hey folks! Good to have you all back! 14:01:11 <marios> o/ ooo 14:01:17 <matbu> o/ 14:01:18 <EmilienM> ccamacho: yeah I missed you too 14:01:19 <florianf> o/ 14:01:20 <sshnaidm> o/ 14:01:24 <ccamacho> ajaj :P 14:01:35 <rasca> o/ 14:01:39 <jistr> hi 14:02:11 <EmilienM> #topic one off agenda items 14:02:16 <EmilienM> #link https://etherpad.openstack.org/p/tripleo-meeting-items 14:02:25 <EmilienM> if you have more topics for this week, please add it 14:02:35 <EmilienM> rasca: floor is yours :) 14:03:13 <rasca> ok, thanks, so basically the topic here is to decide whether to give more exposure to tripleo-quickstart-utils pcakage 14:03:27 <mwhahaha> o/ 14:03:30 <shardy> o/ 14:03:31 <rasca> basically it is a series of tools dedicated to HA 14:03:47 <rasca> to test, apply things like stonith and instance ha 14:03:50 <rasca> and so on 14:03:56 <cdearborn> o/ 14:04:05 <rasca> today this lives under redhat-openstack github repo 14:04:33 <rasca> and despite of the name it is NOT a thing usable just with quickstart deployments 14:04:39 <marios> rasca: why does it not belong in quickstart extras sorry for silly question 14:04:42 <EmilienM> rasca: is it something that could live in tripleo-quickstart-extras? 14:05:44 <rasca> marios, EmilienM, originally it was part of extras, but with the guys from the team we realized that HA was a topic limited to few (basically me) and it was really hard to find out guys for reviews 14:06:03 <rasca> marios, EmilienM, so we decided to move it off and create that redhat-openstack repo 14:06:26 <weshay> 0/ 14:06:27 <rasca> since this was separated I worked to keep the things usable also off-quickstart 14:06:31 <EmilienM> in general, i'm not in favor of using redhat-openstack namespace for tripleo related projects 14:06:47 <adarazs> o/ 14:06:47 <EmilienM> tripleo aims to be 100% upstream and part of OpenStack 14:06:52 <rasca> EmilienM, yes, and this is why I'm raising the topic 14:06:58 <marios> rasca: thanks - perhaps consider lowering barrier to commit if there aren't enough reviewers yet (i mean maybe we say only one +2 14:06:59 <weshay> EmilienM, I think we're hoping to move it an upstream repo 14:07:25 <marios> rasca: and moving it toa completely new repo isn't going to help with reviews unless i miss more of the context 14:07:27 <EmilienM> would it make sense to have it to extras again and give +2 to rasca on the HA bits? 14:07:55 <weshay> EmilienM, that's a valid option 14:08:03 <weshay> imho 14:08:13 <sshnaidm> EmilienM, just automatic merge? 14:08:20 <EmilienM> wat 14:08:24 <matbu> EmilienM: this is something that happens also for upgrade stuff 14:08:37 <EmilienM> is it something we could eventually test? 14:08:48 <trown> the problem was that rasca had to self merge a bunch of stuff essentially 14:08:53 <matbu> EmilienM: but it was improve a bit with new contributor like jistr 14:09:06 <trown> because existing cores did not understand the feature well enough to do timely reviews 14:09:15 <trown> similar problem with upgrades 14:09:54 <rasca> in addition guys, this set of utils is meant to be usable also outside quickstart 14:10:16 <rasca> I'm thinking about someone who wants to test HA on its env, well with this he can 14:10:16 <EmilienM> rasca: is there any validations bits? 14:10:25 <trown> so giving rasca +2 on just the ha parts wont help much... we still need other people to look at it and do good reviews 14:10:28 <rasca> EmilienM, what do you mean? 14:10:38 <EmilienM> rasca: because we also have tripleo-validations, based on ansible 14:10:47 <EmilienM> trown: sure, what do you propose then? 14:10:51 <marios> EmilienM: while we're on the topic, matbu should definitely have +2 as well afaics i mean he has contributed lots to the oooq/extras and the upgrades work there /ci and that should also help? 14:10:51 <shardy> rasca: FWIW I think there's a bunch of stuff in quickstart which should be more easily usable outside of the main quickstart flow 14:11:10 <shardy> so in general making things decoupled so they can be reused by $whatever is a good thing IMO 14:11:22 <matbu> shardy: +1 14:11:22 <weshay> shardy, we're working on that 14:11:32 <trown> marios: not sure this is the best place to bring that up... but I have not seen many reviews 14:11:33 <rasca> shardy, this is exactly why I developed the repo like it is 14:11:41 <weshay> just in a proposal phase atm really 14:11:46 <rasca> and more reviews are going to be merged in that sense 14:11:48 <trown> marios: and +2 is more about high quality reviews than submissions 14:12:05 <marios> trown: ack (yea that was the big thing for another person i had in mind) agree and i just brought it up as the option of giving +2 to alleviate the reviews came up for ooq 14:12:10 <EmilienM> +1 with what trown said ^ 14:12:19 <marios> trown: i mean as well (in mind as well, the reviews #) 14:12:43 <weshay> shardy, myoung|bbl and I will be presenting something to uncouple roles to the group of folks in the prod chain 14:13:08 <shardy> weshay: great, a mail to the list with some details would be awesome too :) 14:13:12 <weshay> ya 14:14:16 <EmilienM> I see a bunch of things that could be used by TripleO directly: 14:14:18 <EmilienM> https://github.com/redhat-openstack/tripleo-quickstart-utils/tree/master/roles/instance-ha 14:14:27 <EmilienM> are they day 2 operations? 14:14:44 <rasca> EmilienM, yes they are 14:15:06 <rasca> EmilienM, for example, as you may know, once you apply instance-ha you lose the ability to do upgrades 14:15:19 <EmilienM> I'm wondering if we could create a repo with ansible tasks for managing tripleo on day 2 14:15:27 <rasca> EmilienM, this role makes you able to apply, then undo (if you need upgrade) and then redo 14:15:28 <EmilienM> it wouldn't be HA focused (or maybe?) 14:16:04 <rasca> EmilienM, I'm focused on ha just because of my everyday job, but yes, a more general thing shuld fit 14:16:09 <EmilienM> the ansible tasks to manage upgrades are in THT though 14:16:29 <EmilienM> if you say that you need to apply some ansible tasks during the upgrade, why not doing that in THT? 14:16:45 <EmilienM> and some logic to whether or not run the playbooks when instance-ha is enabled 14:17:31 <rasca> EmilienM, you're opening a big window here 14:18:03 <rasca> EmilienM, on the stonith side for example, we already started to build something integrated in THT 14:18:16 <rasca> EmilienM, but it is a wider topic IMHO 14:18:43 <EmilienM> well, it's a big topic yes 14:18:56 <EmilienM> but your bits overlap with https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/pacemaker.yaml#L148 for example 14:19:11 <EmilienM> if you say that some tasks needs to be executed during the upgrade process, I think these tasks can live here ^ 14:19:15 <EmilienM> instead of creating a new repo 14:19:32 <rasca> EmilienM, no I'm not saying that 14:19:37 <EmilienM> if that's day 2 operations that are not upgrades, then OK we need to investigate a new repo eventually 14:20:02 <rasca> EmilienM, I just said that once you apply instance-ha you have no guarantee that the upgrade will end file, so you need to undo it first 14:20:13 <EmilienM> right 14:20:18 <rasca> EmilienM, but the instance-ha specific case is something we don't support 14:20:34 <rasca> and, correct me if I'm wrong, we'll never support 14:20:46 <rasca> but someone can be interested into 14:20:47 <EmilienM> there are 3 use cases, right? 14:20:57 <EmilienM> validate-ha is some sort of validation, right? 14:21:03 <rasca> yes 14:21:11 <rasca> we can consider it "apart" 14:21:13 <EmilienM> "This role acts on an already deployed tripleo environment, testing all HA related functionalities of the installation." 14:21:20 <rasca> exactly 14:21:23 <EmilienM> why can't we move these bits into tripleo-validations then? 14:21:34 <EmilienM> we have a git repo for that and good reviewers 14:22:07 <EmilienM> rasca: https://github.com/openstack/tripleo-validations 14:22:15 <florianf> rasca: happy to review ;-) 14:22:38 <EmilienM> for the stonith use case: is it a day 1 operation? (deployment) 14:22:58 <EmilienM> I see: https://github.com/redhat-openstack/tripleo-quickstart-utils/blob/master/roles/stonith-config/tasks/main.yml and to me it sounds like something you want to do at day one 14:23:03 <rasca> EmilienM, I will take a look at the tripleo-validations project 14:23:25 <weshay> rasca, I need to get in there too for some containers work, we can work together a bit 14:23:26 <rasca> EmilienM, this have an explanation 14:24:00 <rasca> EmilienM, we do support stonith, but just from ocata (and the process is, let me say... tricky) 14:24:11 <EmilienM> ok so instance-ha is not supported, validate-ha would be in tripleo-validations and stonith-config sounds like something that could live in THT/puppet 14:24:54 <rasca> EmilienM, since stonith is a dependency of instance-ha I just kept the things easier and wider (supported from kilo to pike) creating the stonith role 14:25:30 <EmilienM> but i thought we didn't support instance-ha 14:25:51 <EmilienM> so if we manage to move validate-ha in tripleo-validations, can we trash the repo? or do you still need it? 14:26:47 <EmilienM> rasca: we just spent 25 min on $topic, let's continue offline... 14:26:54 <rasca> EmilienM, before trashing this repo you need to pass over my body :D or at least I need to be certain that all the things made there can be done with something else 14:26:55 <EmilienM> I'll go to the next topic and give people time 14:27:03 <rasca> EmilienM, sure, no problem at all 14:27:17 <EmilienM> rasca: let's take it offline, mailing-list. Please send a thread if not already done 14:27:24 <EmilienM> and i'll reply with my proposal 14:27:29 <EmilienM> so we have open discussion etc etc 14:27:55 <matbu> rasca: don't forget to mention your past carreer of rugbyman to EmilienM before 14:28:14 <EmilienM> matbu: that's why I moved to Canada, so nobody can find me 14:28:24 <rasca> matbu, past carreer?!?!?!??!?! I'm STILL a player 14:28:24 <EmilienM> ok next topic 14:28:27 <EmilienM> #topic bugs 14:28:32 <EmilienM> #link https://launchpad.net/tripleo/+milestone/pike-2 14:28:40 <EmilienM> do we have outstanding bugs this week? 14:28:59 <matbu> EmilienM: hehe :) rasca sorry man, i thought you were too old ;)) 14:29:00 <EmilienM> I saw we have a bunch of CI alerts 14:29:01 <marios> EmilienM: https://bugs.launchpad.net/tripleo/+bug/1690373 this is not pike 2 but it blocks newton ci 14:29:02 <openstack> Launchpad bug 1690373 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq broken" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) 14:29:23 <EmilienM> marios: yeah this one. I still need to look, maybe today 14:29:30 <marios> EmilienM: well it is targetted at pike 2 i meant it isn't related to Pike .. ack 14:29:32 <EmilienM> marios: do we have any clue before I start looking? 14:29:38 <EmilienM> marios: the target is about milesones 14:29:43 <EmilienM> we want this one closed by Pike-2 14:29:51 <EmilienM> it doens't mean the code is in master (mike) 14:29:54 <EmilienM> (pike) 14:29:58 <matbu> i have 2 backports for a heat bug, idk if it's the right place 14:30:01 <marios> EmilienM: added some notes on the bug and bandini also looked. the case passed last on the 9th so is good to have that to compare to for eliminating the rrors you might see in logs 14:30:04 <shardy> I reopened bug #1687908 as similar issues are still blocking upgrade CI 14:30:05 <openstack> bug 1687908 in tripleo "Could not fetch contents for file:///usr/share/openstack-tripleo-heat-templates/extraconfig/tasks/post_puppet_pacemaker.yaml" [Critical,In progress] https://launchpad.net/bugs/1687908 - Assigned to Carlos Camacho (ccamacho) 14:30:19 <shardy> I've got a patch up which I'm hoping will fix it 14:30:25 <shardy> https://review.openstack.org/#/c/464667/ 14:30:31 <EmilienM> shardy: again? thanks for looking at it 14:30:42 <EmilienM> matbu: ping folks on #heat 14:31:01 <ccamacho> yeahp same issue related with the paths but failing in another stage now 14:31:21 <shardy> matbu: post them and ping me and zaneb, we can probably help with the backport reviews 14:31:35 <EmilienM> sounds like critical things are mostly WIP. Is there any other bug to discuss this week? 14:31:37 <matbu> shardy: EmilienM ack 14:32:22 <EmilienM> #action rasca to send ML about tripleo-quickstart-utils so we keep open discussion going 14:32:33 <EmilienM> #topic projects releases or stable backports 14:32:44 <EmilienM> no update on my side about release / backports 14:32:56 <EmilienM> anything before we move on? 14:33:22 <EmilienM> #topic CI 14:33:43 <EmilienM> so like we said, a bunch of alerts now, folks are working on it for most of them but feel free to join the debug party. 14:33:59 <EmilienM> adarazs, sshnaidm: any quickstart-related update? 14:34:20 <sshnaidm> EmilienM, no, everything works afaik 14:34:21 <adarazs> as far as I understand there's nothing critical directly with quickstart. 14:34:38 <EmilienM> good to know 14:34:44 <adarazs> there are some bigger problems with gate jobs, but they are not quickstart problems. 14:34:58 <EmilienM> adarazs: any reference? 14:35:04 <adarazs> sure, sec, digging it up 14:35:20 <EmilienM> just making sure the problems are seen and someone looking at it 14:35:35 <sshnaidm> https://bugs.launchpad.net/tripleo/+bug/1680195 14:35:36 <openstack> Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,Triaged] 14:35:49 <EmilienM> yeah this one is in our radar already 14:35:50 <adarazs> yeah, that's the most critical 14:35:54 <EmilienM> ok 14:36:10 <EmilienM> I'll spend time on it this week 14:36:24 <EmilienM> anything else before we move on? 14:36:27 <adarazs> https://bugs.launchpad.net/tripleo/+bug/1690132 -- this is newton only but still not good. 14:36:27 <openstack> Launchpad bug 1690132 in tripleo "Newton HA failure: unable to push cib, cib_apply_diff failed" [Critical,Triaged] 14:36:48 <EmilienM> ok also in alert ^ so in our radar too 14:36:54 <EmilienM> ok moving on 14:36:55 <adarazs> that's it. 14:36:59 <EmilienM> adarazs: thx 14:37:01 <EmilienM> #topic specs 14:37:08 <EmilienM> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open 14:37:32 <EmilienM> I see 2 updates in our specs since last meeting 14:37:46 <EmilienM> #link https://review.openstack.org/420878 tempest/mail spec 14:38:02 <EmilienM> #link https://review.openstack.org/462900 Best practices for logging of containerized services 14:38:15 <EmilienM> I guess they are ready for review 14:38:41 <EmilienM> anything to discuss about specs this week? 14:39:13 <EmilienM> okidoki 14:39:15 <EmilienM> #topic open discussion 14:39:46 <EmilienM> last week, shardy flaper87 and I did a presentation about TripleO roadmap 14:39:55 <EmilienM> #link https://docs.google.com/presentation/d/1knOesCs3HTqKvIl9iUZciUtE006ff9I3zhxCtbLZz4c 14:40:12 <EmilienM> read it share it tweet it, and also any feedback is welcome 14:40:21 <EmilienM> I'll probably share it to the ML to make sure folks can see this info 14:40:34 <EmilienM> #action EmilienM to share tripleo project updates slides to ML 14:40:40 <shardy> We also did an onboarding session, slides are here (inc a link to github): 14:40:46 <shardy> #link http://hardysteven.blogspot.co.uk/2017/05/openstack-summit-tripleo-project.html 14:41:00 <EmilienM> oh we can probably do one email with both links 14:41:04 <marios> EmilienM: https://www.openstack.org/videos/boston-2017/project-update-triple0 14:41:06 <EmilienM> shardy: do you mind if I steal it? 14:41:11 <EmilienM> marios: oh I hate videos 14:41:23 <marios> EmilienM: sure but it may be useful for others :) 14:41:23 <EmilienM> I hate my accent 14:41:25 <EmilienM> anyway 14:41:34 <EmilienM> is there anything else for this week? 14:41:34 <shardy> EmilienM: sure, although I already linked the slides for the onboarding, there was a thread asking for them for all projects last week 14:41:50 <EmilienM> shardy: ok great 14:42:16 <EmilienM> have a great and happy week folks 14:42:31 <EmilienM> #endmeeting