14:00:15 #startmeeting tripleo 14:00:16 Meeting started Tue Sep 4 14:00:15 2018 UTC and is due to finish in 60 minutes. The chair is jaosorior. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:17 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:17 #topic agenda 14:00:19 * Review past action items 14:00:20 The meeting name has been set to 'tripleo' 14:00:21 * One off agenda items 14:00:23 * Squad status 14:00:25 * Bugs & Blueprints 14:00:27 * Projects releases or stable backports 14:00:29 * Specs 14:00:31 * open discussion 14:00:33 Anyone can use the #link, #action and #info commands, not just the moderatorǃ 14:00:35 Hey folks! who's around? 14:00:38 ahoy 14:00:48 \o 14:00:53 o/ 14:00:54 o/ 14:00:54 o/ 14:00:58 o/ 14:01:01 o/ 14:01:02 o/ 14:01:13 o/ 14:01:20 \o/ 14:01:36 hi 14:01:47 o/ 14:01:52 o/ 14:02:00 o/ 14:02:00 o/ 14:02:13 o/ 14:03:14 #topic review past action items 14:03:23 there was one: jaosorior to update the wiki and tripleo docs to refer to the squads and their respective etherpads 14:03:41 So... I updated the wiki to mention the squads 14:03:51 #link https://wiki.openstack.org/wiki/Meetings/TripleO 14:04:07 And added the squads to your docs too: 14:04:09 #link http://tripleo.org/contributor/index.html#squads 14:04:15 o/ 14:04:49 o/ 14:04:53 And that's all on that action. 14:05:04 #topic one off agenda items 14:05:09 #link https://etherpad.openstack.org/p/tripleo-meeting-items 14:05:15 hey folks! o/ 14:05:34 o/ 14:05:35 So, due to a lot of folks being in the PTG, there will be no meeting next week 14:06:01 I'll send a reminder of this to the mailing list 14:06:15 #topic PTG agenda 14:06:23 There is a proposed schedule for the PTG: 14:06:29 #link https://etherpad.openstack.org/p/tripleo-ptg-stein 14:06:42 but there is still time for folks to request changes and topics. So, please do. 14:07:09 thanks jaosorior for bringing up an agenda ! 14:07:51 jaosorior: one note: if we're to do a presentation about the validation framework, I'll be remote from EMEA 14:08:15 thanks jaosorior this is looking good 14:08:44 Tengu: we can make sure to enable a bluejeans session so you can participate 14:09:05 jaosorior: securing that would be great - if possible at a nice emea-compatible time :) 14:09:08 I believe such a bridge was available last year. EmilienM was it just your bridge that was put up for folks to join? 14:09:21 yeah 14:09:27 I'll take care of that no worries 14:09:35 thanks EmilienM 14:09:53 If anyone can take a conference call micro it would be great 14:09:59 ah, it's the UX/UI part. just found the schedule. 14:10:14 Tengu: I believe your topic is schedules for thursday in the evening, so it would be morning for oyu. 14:10:17 URGENT TRIPLEO TASKS NEED ATTENTION 14:10:19 https://bugs.launchpad.net/tripleo/+bug/1786764 14:10:20 https://bugs.launchpad.net/tripleo/+bug/1787910 14:10:21 https://bugs.launchpad.net/tripleo/+bug/1789898 14:10:21 https://bugs.launchpad.net/tripleo/+bug/1790199 14:10:21 Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea) 14:10:22 https://bugs.launchpad.net/tripleo/+bug/1790489 14:10:22 https://bugs.launchpad.net/tripleo/+bug/1790490 14:10:23 Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) 14:10:24 Launchpad bug 1789898 in tripleo "check failing due to missing python-tripleoclient" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) 14:10:25 Launchpad bug 1790199 in tripleo "fs035: openstack overcloud deploy returns error code without any signs or error" [Critical,Triaged] 14:10:27 Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New] 14:10:28 Launchpad bug 1790490 in tripleo "failure with collect-logs : Verify Sphinx build" [Critical,Triaged] 14:10:29 jaosorior: more or less yep. 14:10:54 #topic Logging 14:10:59 jbadiapa this one is yours 14:11:24 jaosorior, thanks 14:11:37 tripleo logging status is not in a very good status 14:11:46 mwhahaha: thanks for the quick review! 14:12:01 there are several services that are not properly configured 14:12:33 so, I would like to enable fluentd on all the scenarios if that is possible 14:12:53 jbadiapa: So I'm not sure we should enable it on all scenarios 14:13:06 i dont' think we want to get into a place where fluentd suddenly breaks everythign 14:13:15 jbadiapa: is fluentd going to last? I think I heard that one might be dropped in favor of something else. 14:13:23 additionally we don't have a native way to configure fluentd endpoint 14:13:42 i think this is a larger issue without having a complete logging solution that is managable via tripleo 14:13:59 there are couple of things here 14:14:00 that includes both agent and server 14:14:18 we should maybe use container standard logging with journald+rsyslog... 14:14:42 first, no matter where we going we need to have a proper logging solution, currently the solution is fluentd 14:15:35 but it is not ok at all, I meant there are several services that are not currently configured 14:15:51 and others they are not configured at all 14:16:19 BTW, enable fluentd would be temporally 14:16:24 jbadiapa: do we have any documentation around what is required to properly integrate a service with fluentd? 14:16:57 mwhahaha, I dont think so 14:16:58 if it' snot permanent, it's just going to break again and even just enabling it doesn't mean we have testing 14:17:12 jbadiapa: so we need to resolve that first if we expect folks to integrate with fluentd 14:19:08 ok 14:19:44 Merged openstack/tripleo-heat-templates master: Add reflection of RpcPort to health checks https://review.openstack.org/583629 14:19:45 Merged openstack/python-tripleoclient stable/rocky: Use tripleo-admin user for minor updates https://review.openstack.org/599364 14:20:36 jbadiapa: so we need fluentd integration docs, need a way to configure/deploy fluentd somewhere (undercloud?) in order to have the scenarios log to a central place, and then we'd need to update the CI framework to support pulling logs from this central place rather than each node 14:20:36 I agree that we need to doc how everything should be configured and what we expect form the different services 14:21:05 IMHO, a centralized logging on the undercloud could improve the overall CI logging but it's not a small undertaking 14:21:24 jbadiapa: what's the plan for Stein? is it still to use fluentd or are we switching to rsyslog? 14:21:27 how would that work if we go to all-in-one for the scenario jobs? 14:21:58 jbadiapa: can you fluentd to localhost? 14:23:06 jaosorior, the idea is adding rsyslog for stein and deprecate fluentd on the next release 14:23:22 I guess we can work that out, I just wondered given the PTG topic of reworking the scenario jobs to use only one node 14:24:13 shardy: I guess it shouldn't be too problematic if the server that the services are sending logs to is in the same host. 14:24:31 shardy, It would do but I havent tested TBH 14:24:52 Ok cool, something to keep in mind I guess 14:25:33 mwhahaha, sorry but I'm not sure what you meant 14:26:03 jbadiapa: if you have a single node, can you configure the services to send to a local fluentd instance? 14:26:19 * mwhahaha isn't that familar with the general configuration expected with fluentd 14:26:52 Sorin Sbarnea proposed openstack/tripleo-quickstart master: Avoid silent parsing failure of provisioned.csv https://review.openstack.org/595795 14:27:39 mwhahaha, you don't need to send the logs to nowhere, currently the gate is not sending the log anywhere 14:28:01 it wouldn't be nowhere, it would be localhost 14:28:07 anyway i guess that's a quesiton for another time 14:29:57 jbadiapa: anything else you wanna bring up on this topic? 14:30:27 well, I guess that enable fluentd on all scenarios it not accepted, but at least it's necessary on the scenario we are currently working 14:31:22 what scenario is that? 14:32:21 It depends on the service 14:33:24 and then disable it, otherwise is impossible to be checked on the gate 14:33:49 we can move on, I'll try to explain it on the mailing list 14:33:54 understood, gotta test it out somehow. Is using depends-on enough? 14:33:56 lets do that 14:34:08 #topic Rocky RC2 14:34:11 mwhahaha: ^^ 14:34:12 #link https://launchpad.net/tripleo/+milestone/rocky-rc2 14:34:23 so we haven't yet released RC2, we had some CI issues last week and such 14:34:36 Hopefully this week we can land all the bits, i k now we had a few critical open bugs 14:35:04 I just went through and moved a bunch of bugs that were < Critical but targeted to RC2. Please make sure to target any new stuff to stein-1 unless it's specific to the RC 14:35:31 I'm going to go through the bugs and kickout anything not specifically rocky targeted later today 14:35:51 https://review.openstack.org/#/c/598167/ is imho a blocker 14:35:53 Please let us know if there are any major Rocky related items otherwise we'll cut RC2 and move on 14:35:56 I couldn't deploy rocky without it 14:36:10 +1 14:36:11 yatin -1 it but we need to approve it asap 14:36:18 owalsh: ^ you ok? 14:36:20 Can we get some nova/ironic folks to eyeball it? 14:36:24 can confirm, same issue on my side 14:36:28 It seems like a placement regression or something 14:36:31 if it's only deprecated 14:36:47 mwhahaha: I've added a comment in order to explain what I had on rocky 14:36:49 I checked and the referenced patch is not in stable/rocky 14:37:06 yeah, it's on master 14:38:00 EmilienM: https://review.openstack.org/598167 is required in rocky? 14:38:08 Martin André proposed openstack/tripleo-heat-templates master: WIP Refactor openshift services for composable roles https://review.openstack.org/599618 14:38:33 owalsh: apparently yes. I can't deploy my lab without it because the filters don't properly detect RAM and Disk amount 14:38:37 owalsh: well for me it was required 14:38:47 I couldn't deploy (this morning) my overcloud 14:39:05 there's some trickery in there - Ram is detected as 0MB, same for Disk.... 14:39:41 and as it requires at least 4GB for the ram (and some more disk than the 0MB detected), it prevents deploy :(. 14:40:20 so sounds like we need to track down that issue 14:40:26 Derek Higgins proposed openstack-infra/tripleo-ci master: [WIP] Testing ansible-ml2 job https://review.openstack.org/582294 14:40:27 yes i agree if it's real that's a blocker for rc2 14:40:38 anyway that's it on the RC2 front, let's make sure rocky works :D 14:40:51 thanks mwhahaha 14:41:08 #topic FYI: Blueprint to use newer ceph versions during Stein (fultonj) 14:41:16 fultonj: ^^ 14:41:35 jaosorior: yes, so we have a blueprint 14:41:42 to test newer versions of ceph during stein 14:41:49 EmilienM: didn't think it was necessary, the nova patch is on master only. Either way +1 from me. 14:41:50 wanted to let people know the plan via the blueprint 14:42:04 jaosorior: and wanted you to ack 14:42:21 probably doesn't need a full discussion at PTG 14:42:26 so i bring it up here 14:42:29 #link https://blueprints.launchpad.net/tripleo/+spec/ceph-mimic-nautilus 14:42:36 thanks fultonj 14:42:49 sounds good 14:42:53 thanks 14:43:22 #topic Squad status 14:43:24 ci 14:43:26 #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting 14:43:28 upgrade 14:43:30 #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status 14:43:32 containers 14:43:34 #link https://etherpad.openstack.org/p/tripleo-containers-squad-status 14:43:36 integration 14:43:38 #link https://etherpad.openstack.org/p/tripleo-integration-squad-status 14:43:40 ui/cli 14:43:42 #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status 14:43:44 validations 14:43:46 #link https://etherpad.openstack.org/p/tripleo-validations-squad-status 14:43:48 networking 14:43:50 #link https://etherpad.openstack.org/p/tripleo-networking-squad-status 14:43:52 workflows 14:43:54 #link https://etherpad.openstack.org/p/tripleo-workflows-squad-status 14:43:56 security 14:43:58 #link https://etherpad.openstack.org/p/tripleo-security-squad 14:44:00 edge 14:44:02 #link https://etherpad.openstack.org/p/tripleo-edge-squad-status 14:44:27 Any updates from any squad? 14:45:00 there's still the validation framework spec to validate :) 14:45:22 there's an open discussion regarding some wording/features, would be great to get that sorted out. 14:45:36 (so linked to the validation team I guess mostly) 14:46:12 #topic bugs & blueprints 14:46:14 #link https://launchpad.net/tripleo/+milestone/rocky-rc2 14:46:16 #link https://launchpad.net/tripleo/+milestone/stein-1 14:46:18 For Rocky we currently have 1 blueprints and about 36(+23) open Launchpad bugs. 14:46:20 For Stein we currently have 27 blueprints open in Launchpad. 14:46:22 Bugs: 36 rocky-rc1, 740 (+26) stein-1. 103 (-3) open Storyboard bugs. 14:46:24 #link https://storyboard.openstack.org/#!/project_group/76 14:46:26 Tengu: ack, gotta re-read your validations spec. 14:46:34 jaosorior: thanks :) 14:47:07 Sorin Sbarnea proposed openstack/tripleo-quickstart master: Remove duplicate dictionary keys https://review.openstack.org/597547 14:48:26 #topic projects releases or stable backports 14:48:40 I guess this was already covered by mwhahaha's topic. 14:49:05 #topic specs 14:49:07 #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open 14:49:18 any other specs that need reviews besides Tengu's ? 14:49:29 either way, please review the open ones in the link provided above ^^ 14:50:42 #topic open discussion 14:50:48 Anything else folks want to bring up to the meeting? 14:52:38 Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Avoid fatal task execution when falling back to QEMU https://review.openstack.org/597523 14:54:14 alright folks! 14:54:16 thanks for joining 14:54:22 Merged openstack/python-tripleoclient master: Add fixtures instead mocks https://review.openstack.org/581342 14:54:30 see you at the PTG, else, in two weeks 14:54:33 #endmeeting