14:00:18 <EmilienM> #startmeeting tripleo 14:00:18 <openstack> Meeting started Tue Jul 18 14:00:18 2017 UTC and is due to finish in 60 minutes. The chair is EmilienM. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:19 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:20 <openstackgerrit> Ronelle Landy proposed openstack-infra/tripleo-ci master: WIP: Add settings for ovb in rdocloud https://review.openstack.org/480950 14:00:21 <openstack> The meeting name has been set to 'tripleo' 14:00:27 <EmilienM> #topic agenda 14:00:34 <EmilienM> * review past action items 14:00:36 <EmilienM> * one off agenda items 14:00:38 <EmilienM> * bugs 14:00:40 <EmilienM> * Projects releases or stable backports 14:00:42 <EmilienM> * CI 14:00:44 <EmilienM> * Specs 14:00:46 <EmilienM> * open discussion 14:00:48 <EmilienM> Anyone can use the #link, #action and #info commands, not just the moderatorǃ 14:00:50 <EmilienM> hello, who is around today? 14:00:56 <mwhahaha> hi2u 14:00:57 <marios> hello 14:01:09 <jistr> hi 14:01:11 <cdearborn> o/ 14:01:12 <ccamacho> hi 14:01:23 <chem> o/ 14:01:30 <marios> nice one on the meetbot EmilienM will it do all the logging as usual? 14:01:31 <weshay> o/ 14:01:37 <beagles> o/ 14:01:39 <EmilienM> marios: yup 14:01:56 <EmilienM> #topic review past action items 14:02:04 <EmilienM> gfidente & EmilienM to figure out where if whether or not we package ceph-ansible in RDO: not sure about status 14:02:08 <jrist> o/ 14:02:20 <EmilienM> gfidente to add ceph-ansible to multinode-scenario001-container job: postponed (par of the agenda today) 14:02:27 <dprince> hello 14:02:29 <EmilienM> sshnaidm to add missing oooq jobs in oooq-extras which are already gating somewhere in tripleo: done! 14:02:44 <EmilienM> gfidente: any update on ceph-ansible packaging? 14:03:46 <numans> hi 14:03:47 <saneax> o/ 14:03:50 <shardy> o/ 14:03:52 <EmilienM> let's move around the next topic is with gfidente too 14:03:53 <EmilienM> #topic one off agenda items 14:03:59 <EmilienM> #link https://etherpad.openstack.org/p/tripleo-meeting-items 14:04:07 <adarazs> o/ 14:04:11 <EmilienM> gfidente: you around to discuss about ceph-ansible? 14:04:49 <EmilienM> ok, let's move forward 14:04:55 <EmilienM> numans: please go ahead! 14:05:14 <numans> hi, nothing much to discuss 14:05:24 <numans> just requesting to review some of the OVN patches 14:05:47 <florianf> o/ 14:05:48 <EmilienM> numans: great - any progress on testing it? 14:06:23 <numans> EmilienM, for this review - https://review.openstack.org/#/c/483492/, I have used another patch to test it https://review.openstack.org/#/c/483494/ 14:06:25 <EmilienM> numans: plus, both patches don't pass OVB, not sure our reviewers will +2 right away 14:06:34 <atoth> o/ 14:06:48 <numans> EmilienM, I will have a look into that. 14:07:03 * gfidente o/, we can get to it later 14:07:07 <EmilienM> numans: it would be great to make progress on the scenarios, to have permanent ovn testing coverage 14:07:13 <EmilienM> gfidente: ok 14:07:29 <numans> EmilienM, right. I am just waiting to see if i can use scenario005 14:07:56 <EmilienM> numans: i think scenario005 will keep composable HA architecture, so go ahead and create scenario007 I think 14:08:00 <numans> after which i will submit the patch in t-h-t to use it based on the discussion we had yesterday 14:08:11 <numans> EmilienM, ack. 14:08:19 <EmilienM> numans: I don't think we should, see my last reply on openstack-dev (yesterday) 14:08:37 <EmilienM> numans: scenario005 consumes 2x more than a classic multinode in term of infra resources 14:08:39 <numans> i missed it, i will have a look 14:08:42 <EmilienM> I'm not sure we want that 14:08:44 <EmilienM> ok good 14:08:56 <EmilienM> numans: let's have scenario007 in place this week, finally 14:09:03 <EmilienM> so we can have ovn coverage 14:09:06 <numans> sure. Will do that 14:09:15 <EmilienM> numans: I can help to setup things, just ask me/ping me anytime 14:09:18 <numans> EmilienM, but not very sure on how CI would trigger it 14:09:24 <EmilienM> numans: I'll take care of that 14:09:32 <numans> sure. I will first submit the t-h-t patch for that 14:09:34 <EmilienM> #action EmilienM to prepare scenario007 in project-config 14:09:49 <numans> EmilienM, thanks 14:09:49 <EmilienM> #action numans to prepare scenario007 in THT / oooq 14:09:59 <EmilienM> gfidente: go ahead 14:10:09 <gfidente> EmilienM sorry about late response earlier 14:10:18 <gfidente> so 1) status of ceph-ansible in rdo: it won't get there 14:10:49 <gfidente> instead the agreement was that we should have gating of ceph-ansible changes using tripleo, this takes a while 14:11:07 <gfidente> until then, what we can do, is have a temporary submission in tripleo which enables use of the sig -testing repo 14:11:19 <gfidente> so that when ceph-ansible has interesting changes, we build a test rpm and test it in tripleo 14:11:26 <gfidente> and if it passes, we promote that to -release 14:11:30 <openstackgerrit> Arx Cruz proposed openstack/tripleo-quickstart-extras master: Enable tempest on scenario001-multinode job https://review.openstack.org/482087 14:11:59 <gfidente> long term we probably just want to automate the above process 14:12:08 <EmilienM> gfidente: wfm 14:12:18 <EmilienM> gfidente: go ahead with multinode vs ovb 14:12:25 <gfidente> also, until then, we don't switch existing jobs but only try to convert a scenario 14:12:32 <gfidente> EmilienM ok thanks 14:12:47 <gfidente> this in theory is possible with a small change https://review.openstack.org/#/c/479288/ 14:12:57 <gfidente> but ceph-ansible doesn't permit use of local filesystem as backend for osds 14:13:02 <gfidente> it needs block devices 14:13:07 <openstackgerrit> Arx Cruz proposed openstack/tripleo-quickstart-extras master: Enable tempest on scenario001-multinode job https://review.openstack.org/482087 14:13:17 <gfidente> so we probably can't run the ceph-ansible scenarios in multinode? 14:13:23 <gfidente> or is it possible to add disks there? 14:13:40 <shardy> gfidente: can we point it a loopback device or something just for testing? 14:13:56 <EmilienM> yes ^ 14:14:07 <EmilienM> why can't we just add the feature to ceph-ansible? 14:14:12 <gfidente> EmilienM :D 14:14:14 <EmilienM> puppet-ceph was able to do that 14:14:18 <EmilienM> you're kidding me? lol 14:14:36 <EmilienM> we switched to ceph-ansible because ceph guys wanted it 14:14:37 <shardy> yeah makes me wonder what happens on upgrade from puppet managed ceph too 14:14:42 <EmilienM> and now we have feature limitation to test it in CI 14:14:52 <openstackgerrit> Arx Cruz proposed openstack/tripleo-quickstart-extras master: Set run_tempest default value to false https://review.openstack.org/482087 14:14:54 <gfidente> it actually has other additional limitations too 14:14:56 <shardy> I assume most folks have dedicated disks, but are we sure absolutely everyone does? 14:14:58 <gfidente> not the right forum though I think 14:15:14 <EmilienM> I'm not sure we want to use OVB to test this scenario 14:15:28 <gfidente> but shardy for upgrades, it does have a playbook which can convert non-containerized into containerized 14:15:34 <fultonj> ceph-ansible had the ability to use directories as a backend but then it was removed 14:15:35 <gfidente> shardy this was actually one of the motivatos 14:15:44 <EmilienM> fultonj: why? 14:15:45 <shardy> gfidente: I mean upgrade when there is no spare block device 14:15:59 <fultonj> by design in an effort to drop support for an unsupporte configuration 14:16:19 <fultonj> the code to do it was literally taken out 14:16:35 <EmilienM> so what's the plan now? 14:16:38 <shardy> fultonj: understood, I guess we'll have to look at CI specific workarounds then 14:16:49 <gfidente> fultonj or bring back the code into it? 14:16:52 <EmilienM> I'm not sure we have CI resources to run OVB jobs for that 14:17:16 <fultonj> my vote is to make virtual block devices, quickstart has the means to do this with one flag 14:17:33 <gfidente> fultonj wait for multinode that isn't possible 14:17:36 <shardy> fultonj: that doesn't help in CI because quickstart doesn't create the VMs for multinode jobs 14:17:44 <EmilienM> quickstart is not customer experience 14:18:02 <gfidente> EmilienM do we have existing code I could look at which runs on the subnode 14:18:11 <gfidente> EmilienM so I can create the loopback device ? 14:18:38 <EmilienM> gfidente: yes, we do - we can show this code after the meeting 14:18:46 <fultonj> quickstart aside, i hope a workaround is possible (sorry i don't know one) in ci to use the virtal disks as that's what any serious ceph deploy would use 14:18:51 <gfidente> EmilienM great thanks, I guess that'd be my next action 14:19:12 <openstackgerrit> Arx Cruz proposed openstack-infra/tripleo-ci master: Add validate-tempest role in overcloud-validate tag https://review.openstack.org/484627 14:19:13 <fultonj> thanks 14:19:13 <shardy> fultonj: ack, yeah we're just a little constrained in the CI environment that's all 14:19:31 <fultonj> understood, thank you guys 14:19:33 <EmilienM> right, we try to keep OVB for the things we need baremetal, like ironic/nova, etc 14:19:37 <gfidente> thanks guys "D 14:19:50 <openstackgerrit> Mehdi Abaakouk (sileht) proposed openstack/tripleo-heat-templates master: aodh: add gnocchi_external_project_owner config https://review.openstack.org/484816 14:19:57 <EmilienM> gfidente, fultonj: let's discuss after the meeting on tech details 14:19:59 <jistr> gfidente: puppet-cinder does something similar https://github.com/openstack/puppet-cinder/blob/master/manifests/setup_test_volume.pp 14:20:21 <jistr> loopback+lvm, so i guess you only need loopback? 14:20:28 <openstackgerrit> Arx Cruz proposed openstack/tripleo-quickstart master: Enable tempest on scenario001-multinode job https://review.openstack.org/484788 14:20:30 <jistr> anyway yea we can discuss after, sorry 14:20:34 <shardy> losetup ftw 14:20:41 <EmilienM> :) 14:20:42 <shardy> well, it works in devstack ;) 14:20:45 <jistr> :) 14:20:50 <EmilienM> no! it's not friday please 14:20:55 <shardy> :D 14:20:57 <gfidente> friday is ceph-ansible 14:21:00 <EmilienM> gfidente: anything else? 14:21:13 * EmilienM keeps his energy for Friday 14:21:19 <gfidente> EmilienM well if people interested in integration wants to look at the submissions 14:21:35 <gfidente> https://review.openstack.org/#/c/482500/ < that and below, it'd be nice :D 14:21:44 <gfidente> that's all thanks :D 14:22:06 <EmilienM> gfidente: cool - if you need reviews, also you can send an email to the ML, detailing what you've done, and folks can look the patches 14:22:17 <gfidente> EmilienM ack 14:22:24 <EmilienM> #topic bugs 14:22:27 <marios> EmilienM: i added some review requests. in particular /#/c/479886/ if folks agree we should do that and should we add for all the services? for /#/c/481635/ kind of falls under validations and is feedback a user (checking for the disable_upgrade_deployment flag) which i initially proposed on tripleo-common but i like the client version better. grateful for any comments and reviews thanks. 14:22:33 <marios> EmilienM: thanks :) 14:22:42 <EmilienM> #undo 14:22:42 <openstack> Removing item from minutes: #topic bugs 14:23:09 <EmilienM> marios: you have a problem with URLS :D 14:23:21 <marios> :) nah i listed them in the etherpad 14:23:36 <openstackgerrit> Giulio Fidente proposed openstack/tripleo-heat-templates master: Convert scenario001-multinode-containers job to ceph-ansible https://review.openstack.org/479288 14:23:40 <EmilienM> ah, there is a third topic ok 14:23:44 <marios> EmilienM: https://etherpad.openstack.org/p/tripleo-meeting-items here i mean 14:23:53 <EmilienM> yeah I missed the third (late) topic 14:24:00 <marios> EmilienM: ack my apologies 14:24:13 <EmilienM> no worries 14:24:27 <EmilienM> marios: so what's the problem? please bring some context 14:24:40 <marios> EmilienM: i added some review requests. in particular /#/c/479886/ if folks agree we should do that and should we add for all the services? for /#/c/481635/ kind of falls under validations and is feedback a user (checking for the disable_upgrade_deployment flag) which i initially proposed on tripleo-common but i like the client version better. grateful for any comments and reviews thanks. 14:25:07 <marios> EmilienM: not sure it is worth going into here. there are bugs attached to the reviews for more info 14:25:53 <EmilienM> marios: so you don't want to discuss about it? 14:26:20 <marios> EmilienM: thats fine unless there are questions, otherwise as I said just review requests thanks 14:26:47 <EmilienM> marios: next time, just bring a bit more context for those who're not familiar with this work 14:27:03 <marios> EmilienM: sure thing :D 14:27:33 <shardy> EmilienM: I think the summary is should we remove the packages on the host for containerized services on upgrade 14:27:34 <EmilienM> #action folks who understand marios can review his work about 479886 etc 14:27:42 <shardy> adding a flag to allow it seems reasonable to me 14:27:49 <marios> EmilienM: :( wow man why? 14:27:53 <shardy> although I wonder why the patch mentioned only has neutron and nova in it 14:28:06 <EmilienM> shardy: ah, now I get it 14:28:20 <EmilienM> shardy: shouldn't it be mandatory when we upgrade? 14:28:34 <EmilienM> I'm asking if our operators have to put this flag manually when they upgrade or not 14:28:48 <shardy> EmilienM: well, leaving the packages on the host could enable easier rollback, but we can argue over the default value later 14:28:51 <shardy> ;) 14:28:57 <EmilienM> ok 14:28:59 <EmilienM> #undo 14:29:00 <openstack> Removing item from minutes: #action folks who understand marios can review his work about 479886 etc 14:29:19 <EmilienM> #æction team to review disable_upgrade_deployment flag feature in container work, see https://review.openstack.org/#/c/479886/ and other patches 14:29:33 <EmilienM> marios: anything else? 14:30:13 <EmilienM> ok moving on 14:30:15 <EmilienM> #topic projects releases or stable backports 14:30:22 <EmilienM> #link http://lists.openstack.org/pipermail/openstack-dev/2017-July/119788.html 14:30:34 <EmilienM> a short mail about next releases for pike ^ 14:30:43 <EmilienM> some details about feature freeze, etc, worth reading I think 14:30:50 <EmilienM> other than that, no updates on release management 14:31:08 <EmilienM> any question on topic? 14:31:25 <EmilienM> #topic CI 14:31:42 <EmilienM> I think I missed #bugs, we'll do after 14:32:01 <EmilienM> adarazs: thanks for the weekly summary sent by email on CI work 14:32:10 <adarazs> no problem :) 14:32:25 <EmilienM> is there anything to discuss now, which is related to CI? 14:32:37 <adarazs> this week I won't be on the CI Squad meeting, so please somebody else lead it/summarize it. 14:32:48 <weshay> k.. np 14:32:48 <EmilienM> ah, we need a chair 14:32:53 <EmilienM> weshay: you ok? 14:32:56 <weshay> sure 14:32:58 <EmilienM> cool, thx 14:33:16 <EmilienM> #action weshay to chair next CI squad meeting & summarize it 14:33:21 <adarazs> thanks weshay! 14:33:44 <EmilienM> #topic bugs 14:33:51 <EmilienM> I skipped this topic, sorry 14:34:01 <EmilienM> do we have outstanding bugs to discuss this week? 14:34:31 <EmilienM> next week, we'll move Low / Medium to Queens-1 milestone 14:34:50 <EmilienM> and keep High / Critical for Pike RCs 14:35:07 <EmilienM> please update your blueprints in Launchpad, it would really help :) 14:35:23 <EmilienM> (when I do it it's terrible ;-)) 14:35:39 <EmilienM> nothing else about bugs this week, let's move 14:35:42 <EmilienM> #topic specs 14:35:47 <EmilienM> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open 14:35:55 <EmilienM> anyone needs to discuss about specs? 14:36:31 <EmilienM> #topic open discussion 14:36:46 <EmilienM> any question or feedback is welcome 14:36:51 <marios> EmilienM: hi, i'd like to bring something up here. not sure what you intended when you wrote 17:27 <@EmilienM> #action folks who understand marios can review his work about 479886 etc 14:36:57 <marios> EmilienM: perhaps as a joke? 14:37:04 <EmilienM> marios: yes it was a joke dude 14:37:13 <shardy> I wanted to ask for feedback on https://review.openstack.org/#/c/483293/ - the idea is to have only one (ansible based) workflow for the service deploy steps 14:37:19 <marios> EmilienM: not sure, but i'd like to point out that this is not conducive to inviting folks that may be new to the project to add things into the meeting, or even speak up ata ll 14:37:30 <marios> EmilienM: if they can expect that the PTL will savage them or make some joke about them 14:37:36 <EmilienM> because like I said, I didn't understand the context on the discussion, you brought up patches without explaining what is it 14:37:50 <EmilienM> marios: sorry 14:37:53 <openstackgerrit> Lukas Bezdicka proposed openstack/tripleo-heat-templates stable/ocata: Ensure yum cache is ready before update https://review.openstack.org/484827 14:37:53 <marios> EmilienM: i am not sure what you meant by that, but it genuinely hurt me and when i said that, youdidn't even acknowledge it, but removed from the agenda 14:38:11 <marios> EmilienM: so regardless of myself, someone seeing that from the PTL during the weekly meeting may not find it that funny 14:38:14 <EmilienM> marios: I didn't mean to hurt or anything 14:38:46 <EmilienM> marios: it wasn't funny 14:38:53 <openstackgerrit> Giulio Fidente proposed openstack/tripleo-heat-templates master: [WIP] Add support for deploying RGW with ceph-ansible https://review.openstack.org/480804 14:39:00 <EmilienM> marios: sorry for that :-( 14:39:06 <marios> EmilienM: ack thanks i felt it necessary to at least say that thanks for the apology accepted 14:39:38 <EmilienM> I just want to keep tripleo-meeting-items a set of topics that we can discuss and we like when topics are understable by everyone so we can participate, that was all my intention 14:40:00 <EmilienM> I hope we can make tripleo-meeting-items not just a list of patches to review 14:40:12 <EmilienM> but also a discussion we can have together 14:40:25 <EmilienM> so again, sorry I didn't want to hurt at all 14:40:38 <marios> EmilienM: ack thanks, lets move onto more important things i believe shardy had an item here 14:40:51 <matbu> i wanted to bring a little technical discussion here, may be not the right place but 14:40:59 <shardy> yeah just a request for validation of the approach, it needs a rebase but it did pass CI on the previous revision 14:41:10 <EmilienM> shardy: so you want to deploy in one step, it's interesting 14:41:22 <shardy> if folks are OK with it, I'll rebase and we can aim to completely remove the old puppet deploy steps architecture for pike 14:41:40 <EmilienM> shardy: how do you deal with clustering, etc? 14:41:43 <shardy> EmilienM: No we still have the steps, but they're not duplicated between the docker+puppet and pure puppet architecture 14:41:54 <shardy> so we only have one codepath to maintain 14:41:57 <EmilienM> matbu: it's the right place 14:42:08 <shardy> which will handle baremetal steps and docker steps interleaved 14:42:31 <EmilienM> shardy: ah ok 14:42:38 <gfidente> and workflows 14:43:06 <matbu> ack, so in this review ( https://review.openstack.org/463728 ) i use the std.ssh action from mistral, in order to trigger an ssh remote command for upgrade. I need to tweak a little the ssh config for the mistral user. 14:43:07 <EmilienM> sounds like a good thing, if we can let ansible / workflows manage their own steps 14:43:16 <shardy> gfidente: ah yeah sorry forgot workflows ;) 14:43:25 <gfidente> shardy until we get to run them outside heat 14:43:27 <gfidente> which I am happy with 14:43:32 <itzikb_> EmilienM: hi, can you please look again at https://review.openstack.org/#/c/452521/? 14:43:35 <rook> using scheduling hints i am getting : 14:43:39 <rook> [u'Only 0 nodes are exposed to Nova of 5 requests. Check that enough nodes are in "available" state with maintenance mode off.', u'Only 0 nodes are exposed to Nova of 5 requests. Check that enough nodes are in "available" state with maintenance mode off.'] 14:43:39 <matbu> so i wanted thoughts from workflow guys & others about that ... 14:43:42 <gfidente> but for now heat does the job pretty well for me 14:43:46 <rook> any thoughts? ^^^ 14:44:05 <shardy> matbu: sounds OK short term, but FWIW I was hoping we'd move more towards running ansible on the undercloud via mistral 14:44:25 <shardy> which can then include the upgrade_tasks etc directly 14:44:44 <EmilienM> itzikb_: done 14:44:51 <shardy> matbu: e.g config download can include wrapper playbooks that replace the operator driven upgrade shell scripts 14:46:27 <matbu> shardy: yep, maybe i should do it (i mean the playbook on the nodes, and just re-use the mistral ansible action instead ?) 14:47:03 <shardy> matbu: yeah it might be a little cleaner, and it's similar to what I've been trying for minor updates 14:47:06 <matbu> shardy: because tweaking the mistral user is a bit ugly 14:47:11 <itzikb_> EmilienM: thanks! 14:47:16 <shardy> matbu: I'll try to push some patches later and we can discuss tomorrow 14:47:34 <shardy> about the different approaches, and what is possible for Pike 14:47:40 <matbu> shardy: push patches about this topic or just minor update ? 14:48:07 <shardy> matbu: I'll try to write one which shows how we might run upgrade_tasks from the undercloud vs via the scripts 14:48:19 <shardy> then you can see if it fits with your ideas and we can discuss further? 14:48:33 <matbu> shardy: ack sure 14:48:53 <EmilienM> anything else for this week? 14:49:09 <matbu> btw the ctlplane upgrade works well 14:49:11 <shardy> rook had a question but perhaps we can handle that after the meeting 14:49:14 <matbu> with the cli patch 14:49:15 <EmilienM> oops yes 14:49:50 <gfidente> shardy regarding the step consolidation, steps are still orchestrated in heat there or am I misreading? 14:50:06 <EmilienM> I'll close the meeting and we can continue discussions 14:50:10 <EmilienM> #endmeeting