Tuesday, 2019-05-14

*** dklyle has quit IRC00:11
*** zbr_ has joined #openstack-meeting00:12
*** zbr has quit IRC00:14
*** larainema has joined #openstack-meeting00:22
*** iyamahat__ has joined #openstack-meeting00:33
*** iyamahat_ has quit IRC00:37
*** bbowen has quit IRC00:39
*** psyton has joined #openstack-meeting00:49
*** psyton has quit IRC00:50
*** macza has joined #openstack-meeting01:00
*** psyton has joined #openstack-meeting01:01
*** macza has quit IRC01:04
*** bbowen has joined #openstack-meeting01:05
*** lseki has quit IRC01:12
*** ttsiouts has joined #openstack-meeting01:22
*** enriquetaso has joined #openstack-meeting01:27
*** baojg has joined #openstack-meeting01:27
*** diablo_rojo has quit IRC01:27
*** enriquetaso has quit IRC01:32
*** baojg has quit IRC01:32
*** dklyle has joined #openstack-meeting01:37
*** hongbin has joined #openstack-meeting01:49
*** ttsiouts has quit IRC01:54
*** dklyle has quit IRC01:54
*** apetrich has quit IRC01:57
*** bbowen has quit IRC02:14
*** tpatil has joined #openstack-meeting02:41
*** verdurin has quit IRC02:46
*** macza has joined #openstack-meeting02:48
*** macza has quit IRC02:52
*** whoami-rajat has joined #openstack-meeting02:59
*** ttsiouts has joined #openstack-meeting03:03
*** dtrainor has quit IRC03:10
*** ijw has joined #openstack-meeting03:18
*** iyamahat__ has quit IRC03:20
*** yamahata has quit IRC03:20
*** ttsiouts has quit IRC03:35
*** tpatil has quit IRC03:35
*** igordc has quit IRC03:44
*** tpatil has joined #openstack-meeting03:52
*** ricolin has joined #openstack-meeting03:58
*** shilpasd has joined #openstack-meeting03:59
samPHi all for masakari04:00
samP#startmeeting masakari04:00
*** tashiromt has joined #openstack-meeting04:00
openstackMeeting started Tue May 14 04:00:44 2019 UTC and is due to finish in 60 minutes.  The chair is samP. Information about MeetBot at http://wiki.debian.org/MeetBot.04:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.04:00
*** openstack changes topic to " (Meeting topic: masakari)"04:00
openstackThe meeting name has been set to 'masakari'04:00
samPHi all!04:00
tpatilHi04:00
tashiromtHi04:01
samPtpatil: tashiromt: hi04:01
samP#topic Critical bugs and patches04:01
*** openstack changes topic to "Critical bugs and patches (Meeting topic: masakari)"04:01
shilpasdHi04:01
samPshilpasd: Hi04:01
samPAny critical bugs and pathces to discuss?04:01
tpatilNo04:03
samPThere are some minor fixes. I will review them. Please revirew them if you have time04:04
samPNothing critical from my side04:04
samP#topic Train Work Items04:04
*** openstack changes topic to "Train Work Items (Meeting topic: masakari)"04:04
samP#link https://etherpad.openstack.org/p/masakari-train-workitems04:05
tpatilAdd functional tests for host failure04:06
samPtpatil: thanks04:06
tpatilshilpasd: Do you have any updates?04:06
shilpasdyes04:06
*** diablo_rojo has joined #openstack-meeting04:06
shilpasdhere we are facing one blocker issue04:07
shilpasdCreated instances are not getting cleared off after delete04:07
*** imsurit_ofc has joined #openstack-meeting04:09
shilpasdduring cleanup04:09
tpatilTh problem is: we are forcefully putting down the compute service on the source host, followed by evacuating the instance from source to destination host04:09
samPshilpasd: could you plese share more details04:09
tpatilafter the evacuation is done completely, we are again setting the compute service to up, but the instances are still shown in the virsh list04:10
tpatilon the source host04:10
shilpasdtpatil: thanks for elaborating04:10
tpatilthese will be cleanup either by the periodic tasks in 30 mins or when the compute service on the source host is restarted04:10
tpatilso to write functional tests for host failure, we need ssh access of the compute hosts04:11
samPtpatil: I have the same understaning about cleanup task04:11
*** macza has joined #openstack-meeting04:11
tpatilNeed to collaborate with CI team to see if it can be done04:11
samPgot it. need to restart the host or compute service by ssh04:13
tpatilsamP: Yes04:14
samPI think rather than ssh to node, it is easy to shutdown the node form ansible, since zuul can access to ansible in the test environment04:14
samPHowever, currently we can not use these ansible in opentack infra04:15
*** macza has quit IRC04:16
samPtpatil: as you said, better discuss with infra team about this probelm04:16
tpatilshilpasd: Since the tests will be running on the controller node, we  will need to find out if it's possible to shutdown compute nodes from tests and I think for that we need ssh access04:17
tpatilsamP: Yes, we will discuss about it with the CI team04:17
samPtpatil: Thanks04:17
shilpasdtpatil: will check that04:17
tpatil2. Render graph to show process of recovery workflow04:18
tpatilslow progress, need to find out what additional information will be required to be stored in the progress details so that it can used to render the graph in masakari-dashboard04:18
samPtpatil: got it. thanks04:19
tpatilto store additional information in progress details, we need to make changes to the masakari code but there is a blocker issue due to task flow bug04:20
tpatilwhich is now fixed but the newer version is not yet released04:20
*** yamamoto has joined #openstack-meeting04:20
tpatilshilpasd: Do you have any updates about task flow issue?04:20
shilpasdtpatil: no, will again follow up this week for updated lib for fastners04:21
tpatilshilpasd: Ok04:22
tpatilLast update about devstack support to install hostmonitor04:22
tpatilcode changes are done, testing is in progress04:23
samPtpatil: thanks.04:23
samPThank you for updates. Any update on masakari documentation?04:24
*** pcaruana|afk| has joined #openstack-meeting04:25
tpatilsamP: Not yet started working on these changes04:26
samPtpatil: no problem. I would like to finih this work in train if possible.04:27
tpatilNeed to first list out what changes are needed04:27
tpatil#link : https://bugs.launchpad.net/masakari/+bug/180361004:28
openstackLaunchpad bug 1803610 in masakari "masakari installl docs are not published" [Undecided,New]04:28
tpatilThis bug needs to be fixed04:28
samPtpatil: yes04:29
*** hongbin has quit IRC04:29
samPI think we need to refresh those docs too.04:30
samPNot in hurry, could you please list out needed changes.04:31
tpatilYes, need to focus on operator and developer docs04:31
samPtpatil: thanks04:32
tpatilsamP: I'm on vacation now, will resume office from 27th May. Will check what changes are required post that, is it ok?04:32
samPtpatil: no problem at all.04:32
tpatilsamP: Thanks04:32
samPtpatil: thank you for joining the meeting on your vacation.04:33
samPOK then,04:34
samP#topic AOB04:34
*** openstack changes topic to "AOB (Meeting topic: masakari)"04:34
samPAny other items to discuss?04:34
*** pcaruana|afk| has quit IRC04:35
tpatilNothing from my end04:37
samPtpatil: thanks04:37
samPif no other items to discuss, we can finish the meeting here04:37
tpatilsamP: Ok04:37
samPPlease you openstack-discuss.lists.openstack.org or #openstack-masakari @freenode IRC for futrher discussion04:39
samPThank you all for attending to today's meetin04:39
samPs/meetin/meeting04:39
samP#endmeeting04:39
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"04:39
openstackMeeting ended Tue May 14 04:39:50 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)04:39
openstackMinutes:        http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-05-14-04.00.html04:39
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-05-14-04.00.txt04:39
openstackLog:            http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-05-14-04.00.log.html04:39
*** tpatil has quit IRC04:45
*** janki has joined #openstack-meeting05:01
*** yamamoto has quit IRC05:11
*** yamahata__ has quit IRC05:12
*** yamahata__ has joined #openstack-meeting05:12
*** yamamoto has joined #openstack-meeting05:13
*** radeks_ has joined #openstack-meeting05:19
*** electrofelix has quit IRC05:26
*** ijw has quit IRC05:31
*** ttsiouts has joined #openstack-meeting05:33
*** vishalmanchanda has joined #openstack-meeting05:35
*** yamahata has joined #openstack-meeting05:47
*** imsurit_ofc has quit IRC06:00
*** Luzi has joined #openstack-meeting06:02
*** ttsiouts has quit IRC06:06
*** belmoreira has joined #openstack-meeting06:41
*** slaweq has joined #openstack-meeting06:52
*** diablo_rojo has quit IRC06:58
*** apetrich has joined #openstack-meeting07:01
*** slaweq has quit IRC07:03
*** tesseract has joined #openstack-meeting07:05
*** rcernin has quit IRC07:06
*** hyunsikyang__ has quit IRC07:07
*** hyunsikyang__ has joined #openstack-meeting07:08
*** hyunsikyang has quit IRC07:08
*** hyunsikyang has joined #openstack-meeting07:09
*** pcaruana has joined #openstack-meeting07:12
*** rubasov has quit IRC07:13
*** jesusaur has quit IRC07:13
*** rubasov has joined #openstack-meeting07:13
*** tssurya has joined #openstack-meeting07:25
*** ttsiouts has joined #openstack-meeting07:31
*** tpatil has joined #openstack-meeting07:55
*** sridharg has joined #openstack-meeting07:55
*** ralonsoh has joined #openstack-meeting07:58
*** belmoreira has quit IRC07:59
*** hjwon has joined #openstack-meeting08:01
*** jaewook_oh has joined #openstack-meeting08:07
*** hokeeeeun has joined #openstack-meeting08:07
*** hokeeeeun has quit IRC08:10
*** hjwon has quit IRC08:11
*** jaewook_oh has quit IRC08:19
*** rubasov has quit IRC08:28
*** ttsiouts has quit IRC08:34
*** belmoreira has joined #openstack-meeting08:39
*** priteau has joined #openstack-meeting08:41
*** kopecmartin|off is now known as kopecmartin08:46
*** rubasov has joined #openstack-meeting08:53
*** nnsingh has joined #openstack-meeting08:53
*** electrofelix has joined #openstack-meeting08:54
*** tetsuro has joined #openstack-meeting09:00
*** asmita_ has joined #openstack-meeting09:01
*** asmita_ has quit IRC09:02
*** ttsiouts has joined #openstack-meeting09:10
*** e0ne has joined #openstack-meeting09:12
*** ricolin has quit IRC09:14
*** jesusaur has joined #openstack-meeting09:29
*** yamamoto has quit IRC09:35
*** nnsingh has left #openstack-meeting09:37
*** yamamoto has joined #openstack-meeting09:40
*** tssurya has quit IRC09:41
*** tssurya has joined #openstack-meeting09:41
*** tetsuro has left #openstack-meeting09:43
*** ttsiouts has quit IRC09:44
*** asmita_ has joined #openstack-meeting09:46
*** boxiang has quit IRC09:48
*** belmoreira has quit IRC09:50
*** raildo has joined #openstack-meeting09:53
*** yamamoto has quit IRC09:58
*** yamamoto has joined #openstack-meeting10:02
*** yamamoto has quit IRC10:02
*** gibi is now known as gibi_off10:02
*** yamamoto has joined #openstack-meeting10:03
*** yamamoto has quit IRC10:07
*** iyamahat has joined #openstack-meeting10:11
*** yamahata has quit IRC10:14
*** iyamahat has quit IRC10:17
*** yamamoto has joined #openstack-meeting10:20
*** yamamoto has quit IRC10:24
*** yamamoto has joined #openstack-meeting10:24
*** jbadiapa has joined #openstack-meeting10:28
*** tpatil has quit IRC10:32
*** carloss has joined #openstack-meeting10:51
*** bbowen has joined #openstack-meeting10:51
*** tashiromt has quit IRC10:54
*** bbowen has quit IRC10:56
*** belmoreira has joined #openstack-meeting10:56
*** enriquetaso has joined #openstack-meeting10:56
*** yamamoto has quit IRC11:03
*** njohnston has joined #openstack-meeting11:06
*** e0ne has quit IRC11:31
*** e0ne has joined #openstack-meeting11:31
*** yamamoto has joined #openstack-meeting11:50
*** slaweq has joined #openstack-meeting11:54
*** hyunsikyang__ has quit IRC11:55
*** hyunsikyang__ has joined #openstack-meeting11:56
*** e0ne has quit IRC12:02
*** yamamoto has quit IRC12:04
*** tetsuro has joined #openstack-meeting12:09
*** tetsuro has quit IRC12:11
*** bbowen has joined #openstack-meeting12:16
*** slaweq has quit IRC12:21
*** priteau has quit IRC12:22
*** sridharg has quit IRC12:23
*** mahatic has joined #openstack-meeting12:27
*** janki has quit IRC12:30
*** yamamoto has joined #openstack-meeting12:33
*** sridharg has joined #openstack-meeting12:36
*** slaweq has joined #openstack-meeting12:37
*** slaweq has quit IRC12:41
*** jamesmcarthur has joined #openstack-meeting12:46
*** baojg has joined #openstack-meeting12:48
*** baojg has quit IRC12:48
*** mriedem has joined #openstack-meeting12:49
*** e0ne has joined #openstack-meeting12:50
*** priteau has joined #openstack-meeting12:51
*** ttsiouts has joined #openstack-meeting13:02
*** belmoreira has quit IRC13:02
*** ttsiouts has quit IRC13:07
*** lbragstad has joined #openstack-meeting13:15
*** ijw has joined #openstack-meeting13:17
*** ijw has quit IRC13:20
*** ijw has joined #openstack-meeting13:20
*** baojg has joined #openstack-meeting13:22
*** dmellado has quit IRC13:24
*** dmellado has joined #openstack-meeting13:24
*** ijw has quit IRC13:25
*** ijw has joined #openstack-meeting13:25
*** ttsiouts has joined #openstack-meeting13:26
*** yamamoto has quit IRC13:28
*** ttsiouts has quit IRC13:30
*** jamesmcarthur has quit IRC13:36
*** lseki has joined #openstack-meeting13:38
*** ijw has quit IRC13:43
*** ijw has joined #openstack-meeting13:43
*** baojg has quit IRC13:44
*** ijw has quit IRC13:45
*** ijw has joined #openstack-meeting13:46
*** ijw has quit IRC13:50
*** ijw has joined #openstack-meeting13:50
*** jamesmcarthur has joined #openstack-meeting13:56
*** jamesmcarthur has quit IRC13:57
*** jamesmcarthur has joined #openstack-meeting13:58
*** belmoreira has joined #openstack-meeting13:58
*** dklyle has joined #openstack-meeting14:13
*** Luzi has quit IRC14:18
*** yamamoto has joined #openstack-meeting14:23
*** ijw has quit IRC14:48
*** ijw has joined #openstack-meeting14:48
*** ijw has quit IRC14:55
*** ijw has joined #openstack-meeting14:56
*** jrbalderrama has joined #openstack-meeting14:57
*** ijw has quit IRC14:59
*** ijw has joined #openstack-meeting15:02
*** jrbalderrama has quit IRC15:09
*** ayoung has quit IRC15:10
*** number80 has quit IRC15:16
*** number80 has joined #openstack-meeting15:16
*** wwriverrat has quit IRC15:18
*** wwriverrat has joined #openstack-meeting15:24
*** ttsiouts has joined #openstack-meeting15:27
*** wwriverrat has quit IRC15:28
*** dtrainor has joined #openstack-meeting15:29
*** sridharg has quit IRC15:35
*** ayoung has joined #openstack-meeting15:39
*** hemna has joined #openstack-meeting15:41
*** ijw has quit IRC15:45
*** ijw has joined #openstack-meeting15:46
*** belmoreira has quit IRC15:48
*** dklyle has quit IRC15:50
*** sridharg has joined #openstack-meeting15:51
*** dklyle has joined #openstack-meeting15:52
*** ijw has quit IRC15:54
*** ijw has joined #openstack-meeting15:55
*** e0ne has quit IRC15:57
*** igordc has joined #openstack-meeting15:59
*** ijw has quit IRC15:59
*** ttsiouts has quit IRC16:01
*** macza has joined #openstack-meeting16:01
*** gyee has joined #openstack-meeting16:01
*** igordc has quit IRC16:03
*** ijw has joined #openstack-meeting16:04
*** rsimai is now known as rsimai_away16:09
*** dklyle has quit IRC16:11
*** dklyle has joined #openstack-meeting16:16
*** dklyle has quit IRC16:21
*** ijw has quit IRC16:23
*** dklyle has joined #openstack-meeting16:23
*** ijw has joined #openstack-meeting16:23
*** yamamoto has quit IRC16:24
*** ijw has quit IRC16:24
*** yamamoto has joined #openstack-meeting16:24
*** ijw has joined #openstack-meeting16:24
*** diablo_rojo has joined #openstack-meeting16:26
*** ijw has quit IRC16:29
*** ijw has joined #openstack-meeting16:30
*** yamamoto has quit IRC16:30
*** yamamoto has joined #openstack-meeting16:30
*** vishalmanchanda has quit IRC16:33
*** ijw has quit IRC16:34
*** ijw has joined #openstack-meeting16:35
*** ijw has quit IRC16:35
*** ijw has joined #openstack-meeting16:35
*** ijw has quit IRC16:36
*** yamamoto has quit IRC16:36
*** yamamoto has joined #openstack-meeting16:37
*** ijw has joined #openstack-meeting16:37
*** ijw has quit IRC16:37
*** yamamoto has quit IRC16:37
*** ijw has joined #openstack-meeting16:38
*** yamamoto has joined #openstack-meeting16:38
*** yamamoto has quit IRC16:43
*** ijw has quit IRC16:44
*** jamesmcarthur has quit IRC16:50
*** mattw4 has joined #openstack-meeting16:53
*** jamesmcarthur has joined #openstack-meeting16:56
*** iyamahat has joined #openstack-meeting16:56
*** dklyle has quit IRC16:59
*** armax has quit IRC17:00
*** dklyle has joined #openstack-meeting17:01
*** iyamahat_ has joined #openstack-meeting17:01
*** jamesmcarthur has quit IRC17:01
*** iyamahat has quit IRC17:03
*** tssurya has quit IRC17:06
*** slaweq has joined #openstack-meeting17:06
*** kopecmartin is now known as kopecmartin|off17:07
*** jamesmcarthur has joined #openstack-meeting17:08
*** jamesmcarthur has quit IRC17:11
*** slaweq has quit IRC17:14
*** yamahata has joined #openstack-meeting17:15
*** ayoung has quit IRC17:22
*** ralonsoh has quit IRC17:25
*** rfolco|ruck is now known as rfolco|dentist17:25
*** jamesmcarthur has joined #openstack-meeting17:31
*** slaweq has joined #openstack-meeting17:31
*** _erlon_ has joined #openstack-meeting17:32
*** slaweq has quit IRC17:35
*** sridharg has quit IRC17:36
*** armax has joined #openstack-meeting17:41
*** mattw4 has quit IRC17:48
*** mattw4 has joined #openstack-meeting17:49
*** dklyle has quit IRC17:52
*** ayoung has joined #openstack-meeting17:54
*** jamesmcarthur has quit IRC17:55
*** jamesmcarthur has joined #openstack-meeting17:56
*** e0ne has joined #openstack-meeting17:57
*** ttsiouts has joined #openstack-meeting17:58
*** jamesmcarthur has quit IRC18:01
*** hongbin has joined #openstack-meeting18:04
*** e0ne has quit IRC18:05
*** jamesmcarthur has joined #openstack-meeting18:28
*** ttsiouts has quit IRC18:31
*** ijw has joined #openstack-meeting18:33
*** vishalmanchanda has joined #openstack-meeting18:34
*** ijw has quit IRC18:37
*** ijw has joined #openstack-meeting18:37
*** ijw has quit IRC18:38
*** e0ne has joined #openstack-meeting18:38
*** ijw has joined #openstack-meeting18:38
*** ijw has quit IRC18:38
*** ijw has joined #openstack-meeting18:39
*** ijw has quit IRC18:41
*** ijw_ has joined #openstack-meeting18:41
*** jamesmcarthur has quit IRC18:48
*** ijw_ has quit IRC18:52
*** ijw has joined #openstack-meeting18:53
clarkbanyone else here for the infra meeting?18:59
cmurphyo/19:00
diablo_rojoo/19:00
ianwo/19:00
*** Shrews has joined #openstack-meeting19:00
clarkb#startmeeting infra19:01
openstackMeeting started Tue May 14 19:01:08 2019 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.19:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:01
*** openstack changes topic to " (Meeting topic: infra)"19:01
openstackThe meeting name has been set to 'infra'19:01
clarkblink http://lists.openstack.org/pipermail/openstack-infra/2019-May/006369.html19:01
clarkb#link http://lists.openstack.org/pipermail/openstack-infra/2019-May/006369.html19:01
clarkb#topic Announcements19:02
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:02
clarkbThis didn't end up on the agenda but I think people are still in travel/life spin cycle so don't be surprised if things are a bit slow for a bit19:02
clarkbToday is my first real day back19:02
clarkbGood news is the side of my house is back on and I haven't observed any new laeking \o/19:03
clarkb#topic Actions from last meeting19:03
*** openstack changes topic to "Actions from last meeting (Meeting topic: infra)"19:03
clarkb#link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-05-07-19.01.txt minutes from last meeting19:03
clarkbI had an action to send out a ptg summit forum recap19:04
clarkb#link http://lists.openstack.org/pipermail/openstack-infra/2019-May/006367.html Denver 2019 Summit/Forum/PTG Recap19:04
clarkbAs mentioned feel free to followup with any questions that might arise out of that or need for clarification19:04
clarkbAs expected it was an exhausting trip19:04
clarkbI think cross project efforts like infra and qa etc suffered under the newer ptg format19:05
clarkb(there were far fewer people in our room(s) than in the more siloed efforts)19:05
clarkbbut it was still fairly productive19:05
clarkb#topic Priority Efforts19:05
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:05
clarkbLets dive right in to the fun stuff19:06
clarkb#topic Update Config Management19:06
*** openstack changes topic to "Update Config Management (Meeting topic: infra)"19:06
clarkbWe are super close to finishing the puppet 4 upgrades. After this meeting I want to approve the zuul puppet-4 upgrade change19:06
cmurphy\o/19:06
clarkbthen we can rebase the lists.o.o change without needing to rebase the zuul change19:06
clarkband depending on how zuul change goes can likely get lists.o.o upgrade in today as well19:07
fungisounds good to me19:07
clarkbianw do you want to talk about the ansibling of mirrors and how that will enable us to do tls which should improve docker now or when we get to the official spot for it on the agenda?19:07
fungii'll be around to help fix if it goes sideways19:08
clarkb(I think that effort is fairly tightly cuopled to the config management updates at this point as we are looking at converting to ansible for those nodes and improving docker test job success rates)19:08
ianwi don't mind :)19:08
clarkblets talk about that now then as I think it is useful to have in this context19:09
clarkbyseterday you mentioned the change isn't quite ready for review yet. THis morning I noticed that the puppet jobs are unhappy with it19:09
clarkbAnything we can do to help or will you send up the signal flare when we should look at it?19:09
ianwok; so to catch people up, we only support letsencrypt for .opendev.org domain names, because we use domain authentication and openstack.org ishosted in rax which needs a different path19:10
clarkb#link https://review.opendev.org/#/c/658281/ Ansible deployed mirrors so they can be LetsEncrypted more easily19:11
ianwso to https mirrors we need .opendev.org names for them19:11
ianwmy first thought was that we could do something like point records to the existing mirrors, and deploy certs19:11
ianwbut it turned out to not really be KISS but puppet hacking and just going what felt like the wrong way19:12
ianwso i'd propose we just start a new mirror server in each region and manage it fully via ansible19:12
ianwwhich is ultimately 658281 above19:13
ianwthere's some intermediate steps below that19:13
ianw#link https://review.opendev.org/658930 -- signs fake letsencrypt cert with a fake CA so apache can start unmodified in test environment19:13
ianw#link https://review.opendev.org/652801 -- wires in handlers to be called when letsencrypt renews certs19:13
clarkbplan seems reasonable. Then we can have the new mirrors running with tls enabled then update the site var for zuul to talk to the opendev mirrors instead19:14
ianwfungi would particularly like your eyes on 65893019:14
clarkboh there is a new ps on the handlers change (I'll rereview it)19:14
fungistarred, thanks19:14
ianwyep, and as mentioned in the CL, it should be ok to CNAME the existing mirrors to the new mirrors, it's setup as http/https19:14
clarkbcool that gives us extra betls and suspenders19:15
ianwi wasn't quite sure about the other ports; would they be switched to ssl, or do we need to have a similar thing where ssl is a separate port?19:15
clarkbianw: I think we can safely update the docker ports to be ssl because docker doesn't take urls19:16
clarkbinstead you just give it a flag if it is ok to not do ssl19:16
*** dklyle has joined #openstack-meeting19:16
clarkbbut for the main mirror at 80 we probably want a corresponding 443 so that stuff can opt into https for that19:16
ianwi also considered docker for deployment, but i didn't see where it really fit here.  the mirrors must be in each region specifically; i mean that's the whole point19:16
clarkbthen after the initial switch we can make 80 redirect to 44319:16
ianwthe main thing this is doing is being an afs client, something that doesn't make sense in a container as it installs kernel modules to the host system19:17
clarkbthat and run a very stable web server19:17
clarkbI agree that ansible alone is probably sufficient for this19:17
clarkb(and likely preferable)19:17
ianwat that point, the container is just a webserver with a lot of stuff mapped into it, which doesn't seem any different to just "apt-get install" ing it19:18
ianwright, yeah we don't even have complex cgi dependencies or anything; just that one vhost19:18
clarkbI think if we wanted to build apache or $webserver ourselves containers would make sense but the distro version has been fine so far19:18
fungiyeah, i think we should push to wrap things in containers when it makes sense to do so, and just rely on distro packages for stuff on the server when it doesn't19:18
*** dklyle has quit IRC19:19
*** david-lyle has joined #openstack-meeting19:19
*** pcaruana has quit IRC19:19
fungidon't container merely for containering's sake19:19
ianwcool; that was my only concern, that people might think it needs to be wrapped up19:19
ianwso in short, should be ready for review.  if it's looking good, we can bring up a single node first and check it out, and even switch just one node in with a CNAME from openstack.org and monitor it for a bit19:20
clarkboh I like that idea19:20
*** jamesmcarthur has joined #openstack-meeting19:20
fungime too19:20
clarkb+1 from me on that. I can review the stack while waiting on zuul puppet 4 things to happen19:20
*** slaweq has joined #openstack-meeting19:21
clarkbalright anything else on this or should we move one?19:21
clarkbs/one/on/19:21
ianwnope, that's it, thanks!19:21
clarkb#topic OpenDev19:22
*** openstack changes topic to "OpenDev (Meeting topic: infra)"19:22
clarkbIf you haven't already, please read my ptg summary as it covers some the thoughts we had during the PTG for opendev next steps (largely around how to delegate responsibility for stuff to users19:23
clarkbThe other major todos we have are the cgit cluster cleanup (which I've got on my todo list), just trying to convince myself we don't have a major gitea issue with the 404s mriedem points out before I commit to that19:23
clarkbAlso we want to udpate gitea to 1.8.x but our tests are failing on what appears to be a valid issue pointed out by the tests19:24
clarkbif anyone is able to look at that it would be helpful (I can probably get to it after cgit farm is cleaned up)19:24
clarkbI think we are still likely to be on track for a end of month downtown to do fixup project renames19:25
clarkbwe should prep rename changes for the projects under infra/opendev control that were missed19:25
clarkb(I say all that realizing we are still somewhat short staffed for a bit so we'll get what we get done done)19:26
*** radeks_ has quit IRC19:26
clarkbgraham (kata dev) had good feedback on some gitea UI weirdness19:27
clarkbI'll try to push that feedback upsteram if graham chooses not to19:27
*** jamesmcarthur has quit IRC19:27
clarkbAnything else from the crowd on opendev related items?19:28
*** jamesmcarthur_ has joined #openstack-meeting19:28
clarkbSounds like no. Lets move on19:29
clarkb#topic Storyboard19:29
*** jamesmcarthur has joined #openstack-meeting19:29
*** openstack changes topic to "Storyboard (Meeting topic: infra)"19:29
clarkbdiablo_rojo: still here? I noted last week that there was a painpoints session with good feedback and that you all had a bug cleaning sprint19:29
clarkbAnything else to add re storyboard since the summit/ptg/forum?19:30
fungidiablo_rojo posted a denver summary for it to the... openstack-discuss ml?19:30
fungiyeah that's it19:30
diablo_rojo_phonYes the discuss list19:30
diablo_rojo_phonFinally.19:30
clarkb#link http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006214.html19:30
diablo_rojo_phonBasically nothing was a surprise.19:30
fungiit's prompted some follow-up discussion on the ml as well19:31
diablo_rojo_phonThe largest outcome was that we should do two different onboardings. One for users.19:31
diablo_rojo_phonAnd one for development19:31
clarkbya we've had similar ideas for infra users and admins/devs19:32
fungialso the telemetry team is ready to move their tracking to sb.o.o now that the new maintainers have obtained control over the existing lp projects to be able to wind them down there19:32
diablo_rojo_phonYay!19:33
*** jamesmcarthur_ has quit IRC19:33
diablo_rojo_phonSwift also wants a test migration.19:33
diablo_rojo_phonWhich is on my to-do list for this week.19:33
clarkbis the -dev server happy yet?19:33
clarkbfungi: ^ you were probably the last person looking at that. Did you decide to just start over?19:34
fungias in manually renaming the projects in it? i think we should probably just stop having puppet autocreate project-groups on sb-dev but no i haven't found time to work on that yet19:35
*** yamamoto has joined #openstack-meeting19:35
fungistop having puppet autocreate projects on sb-dev too probably19:35
clarkbthe db migration puppetry is broken currently. Not sure if t hat will impact diablo_rojo_phon's ability to do a test migration for swift19:35
diablo_rojo_phon..sounds like it would19:36
fungiit shouldn't. just have to manually create any projects and project-groups you want to try importing into19:36
fungiwhich would have been the case regardless because they're not flagged to use-storyboard in projects.yaml anyway19:36
diablo_rojo_phonOh okay.19:36
clarkbgot it19:36
diablo_rojo_phonI think that's how I was doing it before.19:36
fungiwhat i'm talking about abandoning is a puppet exec we currently run which creates any missing projects and project-groups in sb-dev for projects marked as using (production) storyboard already19:37
fungithat's of very limited utility anyway given how we've actually been using sb-dev19:37
clarkbshould update the project rename playbook too to remove the renames there?19:38
fungiwe never included them in the project rename playbook, which is why the puppet exec is now broken19:38
fungi(unless someone added it very recently)19:38
*** rfolco|dentist is now known as rfolco|ruck19:39
clarkbmy change (whcih I thought had merged) https://review.opendev.org/#/c/655476/3/playbooks/rename_repos.yaml added it as the alternative fix for this problem19:40
fungiahh19:40
clarkbThe change hasn't merged though (we should really get that in well before we try to do more project renames...)19:40
clarkbI can remove it if you prfer and push a new ps. Then you can remove the puppet exec safely19:40
*** yamamoto has quit IRC19:40
fungiwell, given that the rename process simply updates database rows when it finds matches, i don't think it will hurt (it just won't usually do anything for nonexistent ones)19:40
clarkbok I'll leave it as is then I guess19:41
*** david-lyle has quit IRC19:42
clarkbSounds like that may be it for storyborad19:43
clarkb#topic General Topics19:43
*** openstack changes topic to "General Topics (Meeting topic: infra)"19:43
clarkb#link https://etherpad.openstack.org/p/201808-infra-server-upgrades-and-cleanup Trusty server upgrades19:43
clarkbI'm hoping that I'll be able to dedicate a good chunk of next week to this once I've climed out of the everything else backlog19:43
clarkbwe are very near the end19:43
clarkbfungi said he can delete the old groups servers in the near future too19:44
fungiyup, already on my to do list19:44
clarkbHas anyone been working on the other servers still on the list? if so anything we can help with?19:44
ianwno but i really should look at ask ... i think it's ready to go19:45
clarkbianw: that would be great19:46
clarkb#topic Open Discussion19:46
*** ijw has quit IRC19:46
*** openstack changes topic to "Open Discussion (Meeting topic: infra)"19:46
clarkbWe have ~14 minutes for other topics if we want to keep talking here19:46
fungion opendev topics, did we mention the next batch project rename schedule?19:47
fungilast week we said last friday of may?19:47
clarkbfungi: yup and yup19:47
clarkbthe 31st19:47
clarkbI think we are still on track for that assuming we get the rename fixes merged and can generate the list of things we need to rename19:48
fungicool19:48
fungithat weekend is a holiday in au, ie and nz19:48
clarkbmy kids have a fourth birday on the 2nd19:49
clarkbbut the size of this would be much smaller19:49
clarkbso I'm more comfortable doing it friday and nnot needing to work all weekend like last time19:49
fungiweekend before it is a holiday in uk and us, weekend before that is a holiday in ca ;)19:49
fungiso yeah, gonna be a holiday weekend no matter what19:50
* fungi needs all the countries and religions of the world to sync their holiday schedules for his convenience19:50
clarkbThat would be convenient19:51
clarkbAnything else?19:52
clarkbI'll end the meeting in a minute or two if not19:52
clarkbThank you everyone!19:54
clarkb#endmeeting19:54
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"19:54
openstackMeeting ended Tue May 14 19:54:10 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)19:54
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-05-14-19.01.html19:54
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-05-14-19.01.txt19:54
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-05-14-19.01.log.html19:54
fungithanks clarkb!19:54
diablo_rojo_phonthanks clarkb !19:59
*** trandles has joined #openstack-meeting20:01
*** jmlowe has joined #openstack-meeting20:06
*** tesseract has quit IRC20:06
*** Shrews has left #openstack-meeting20:08
*** jamesmcarthur has quit IRC20:15
*** e0ne has quit IRC20:20
*** bbowen_ has joined #openstack-meeting20:26
*** ttsiouts has joined #openstack-meeting20:28
*** bbowen has quit IRC20:28
*** jamesmcarthur has joined #openstack-meeting20:31
*** bbowen_ has quit IRC20:36
*** jamesmcarthur_ has joined #openstack-meeting20:37
*** jamesmcarthur has quit IRC20:40
*** jamesmcarthur_ has quit IRC20:41
*** vishalmanchanda has quit IRC20:44
*** slaweq has quit IRC20:44
*** jamesmcarthur has joined #openstack-meeting20:47
*** oneswig has joined #openstack-meeting20:47
*** jamesmcarthur has quit IRC20:50
*** rbudden has joined #openstack-meeting20:54
*** jamesmcarthur has joined #openstack-meeting20:55
*** jamesmcarthur has quit IRC20:55
*** jamesmcarthur has joined #openstack-meeting20:55
*** jamesmcarthur has quit IRC20:59
oneswig#startmeeting scientific_sig21:00
openstackMeeting started Tue May 14 21:00:32 2019 UTC and is due to finish in 60 minutes.  The chair is oneswig. Information about MeetBot at http://wiki.debian.org/MeetBot.21:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:00
*** openstack changes topic to " (Meeting topic: scientific_sig)"21:00
openstackThe meeting name has been set to 'scientific_sig'21:00
*** janders has joined #openstack-meeting21:00
jandersgood morning, good evening21:00
jandershow's everyone?21:00
oneswigahoy there :-)21:00
trandleshello21:00
oneswig#link agenda for today https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_May_14th_201921:00
jmlowehello21:01
oneswigGreetings all21:01
*** ttsiouts has quit IRC21:01
oneswigI trust you all made it back in one piece21:01
jandersyes I have, thank you :)21:02
oneswig#topic CERN OpenStack day21:02
*** openstack changes topic to "CERN OpenStack day (Meeting topic: scientific_sig)"21:03
jandersI missed out on the last meeting cause I went to sleep about 830pm and the meeting started at 9pm21:03
oneswigIt's a day to go for registrations... last call21:03
oneswigjanders: it was a very quiet one21:03
jandersbut I'm truly back and on local time. Good conference.21:03
oneswig#link get your CERN tickets here - https://www.eventbrite.com/e/openstack-day-cern-tickets-5434906752421:03
oneswigI agree, really good to see everyone plus some new faces too21:04
*** asmita_ has quit IRC21:04
oneswigThink that's all on the CERN side...21:05
oneswig#topic Denver roundup21:05
*** openstack changes topic to "Denver roundup (Meeting topic: scientific_sig)"21:05
jandersindeed - and having the PTG right after the main conference was a very welcome addition21:05
jandersvery productive21:05
*** raildo has quit IRC21:05
oneswigjanders: did you stay for the duration?  I had to leave on Thursday afternoon, alas21:05
jandersI skipped Saturday, was driving around the Rockies all weekend :)21:06
jandersbut other than that - yes21:06
jandersI did convey the SIG message to the Ironic team together with Mark21:06
janderspretty good reception21:06
trandlesIronic PTG was very useful21:06
jandersdid chat to Miguel about the SDN, too, he's on board, gotta follow this up21:06
oneswigGreat21:07
oneswigtrandles: tried that ramdisk driver yet?21:07
oneswigjanders: which Miguel?21:08
*** martial has joined #openstack-meeting21:09
jandersMiguel Lavalle21:09
oneswigInteresting to hear that.21:11
martialSeems like a quiet meeting today21:12
*** janders_ has joined #openstack-meeting21:12
oneswigLooks like the summit videos have been posted21:13
oneswigAny picks?21:13
oneswig#chair martial21:13
openstackCurrent chairs: martial oneswig21:13
*** b1airo has joined #openstack-meeting21:13
oneswighey martial, glad you made it21:13
jandersmartial: indeed - to a degree where I thought I got kicked out :)21:13
b1airoo/21:13
oneswigI'll admit I'm troubleshooting openvpn weirdness on the side...21:14
martialWell I saw no messages for 3 minutes so I was not sure either21:14
oneswighey b1airo21:14
oneswig#chair b1airo21:14
openstackCurrent chairs: b1airo martial oneswig21:14
b1airoJust realised my calendar is out due to daylight saving changes, hence the lateness21:14
jmlowehttps://threatpost.com/linux-kernel-remote-code-execution/144713/?utm_source=newsletter&utm_medium=Email&utm_campaign=tp_daily_digest_14_5_201921:14
jandersmartial: I saw last week you mentioned good conversations with the RHAT folks21:14
jmlowethat's got my attention ATM21:14
janderswho was that?21:14
jmloweand for extra fun Intel has a new exploit, zombieload21:15
jandersI'm wondering if it's the guys I'm talking to as well, perhaps we can join forces pushing the HPC agenda21:15
martialIt was Erwan Gallen21:15
martialHe is seeing how RedHat can bring efforts and code to help the HPC side21:16
b1airoOpenStack Summit = new high impact vulnerability disclosures21:16
martialI need to follow up so he join our slack too21:16
janderscool! thank you21:17
jandersI think it is a great opportunity for both RHAT and us21:17
jandersso let's keep fighting the good fight21:17
jmloweb1airo: it's the sobering reality after returning from the summit21:17
janders(the subscription model as it stands isn't the best fit for HPC and HPC scale, that's one of the issues I see)21:17
jandershttps://access.redhat.com/security/vulnerabilities/mds?sc_cid=701f20000012nDXAAY&21:17
janders(further to the exploit comments)21:17
martialThey want to help and he is willing to put coders on a problem with the community’s help21:18
martialAnd their code is open source after all21:18
jmlowejanders: those are collectively being called zombieload I think21:18
trandlesoneswig: sorry, was AFK on the phone...not tried ramdisk driver, should be in the next two weeks21:19
jandersjmlowe: agreed21:19
oneswigtrandles: no problem, good to hear it21:19
*** jamesmcarthur has joined #openstack-meeting21:20
trandlesworking through the logistics of getting a couple collaboration accounts set up here for outsiders, nuke-n-pave my testbed to reset for ironic work, etc.21:20
oneswigSound interesting, but careful with your choice of metaphor21:21
b1airo:-)21:21
trandlesHA!21:21
trandlesAlso, saw the latest Singularity root exploit and had a chuckle :P21:21
oneswigThe news over here is we got our first Kata study published21:22
oneswig#link I/O performance of Kata https://www.stackhpc.com/kata-io-1.html21:22
oneswigtrandles: what's that?21:22
martialOh cool21:23
trandleshttps://github.com/sylabs/singularity/releases/tag/v3.2.021:23
trandlestheir CVE link is wrong, you either get the reserved message or a joomla CVE using the same number21:24
oneswighow careless21:24
trandlesscratch that, google brought up 2018-11328 instead of 201921:25
trandles:(21:25
*** dklyle has joined #openstack-meeting21:27
b1airoGah, glad we haven't committed one way or other yet21:27
trandlesseems like Kata has a LOT of I/O optimization to do21:28
b1airotrandles: have you guys considered a Zun plugin for CharlieCloud ?21:29
oneswigtrandles: a lot indeed.  It's moving fast and should get a lot of the low-hanging fruit in the next release.21:29
oneswigtrandles: it has a very long way to go though21:29
trandlesb1airo: not really21:29
oneswigIn other post-summit follow-up, I mailed Jonathan Bryce about supporting a white paper on private / hybrid cloud advocacy for research computing, no response as yet, but I'll follow up21:29
oneswigbrb21:30
trandleswe've been focusing on some Slurm integration21:30
*** jamesmcarthur has quit IRC21:30
jmloweplan9 is always going to kill I/O21:30
trandlesamong other things (like at 100% truly unprivileged container build without docker as a dependency)21:30
*** enriquetaso has quit IRC21:31
trandleswe've been digging into Buildah since it's billed as "unprivileged build" using Dockerfiles...and it relies on setuid binaries and has Docker as a dependency buried in all the golang :(21:31
*** jamesmcarthur has joined #openstack-meeting21:31
b1airoNice o_021:32
jandersbad developers go to dependency hell after they die21:33
oneswigtrandles: similarly I saw that podman claims to offer an unprivileged container runtime but not until the next release of RHEL21:34
trandlesoneswig: in RHEL 8, which is GA21:34
b1airoThe really bad ones go to dll hell :-P21:34
oneswigtrandles: ah is it?  Thanks21:34
trandlesI also heard that RHEL is offering RHEL-based base containers with RHEL 8 that can be redistributed, but haven't confirmed it21:34
jandersnext as in 7.7, 8 or 8.1?21:35
janderssorry scratch that21:35
trandlesI know podman is being looked at here, but I'm not the one doing the looking and haven't followed-up yet21:35
*** jamesmcarthur has quit IRC21:36
*** enriquetaso has joined #openstack-meeting21:37
oneswigtrandles: going from https://developers.redhat.com/blog/2019/02/21/podman-and-buildah-for-docker-users/21:37
oneswig"You can also run Podman from your normal non-root user in Podman 1.0 on Fedora. RHEL support is aimed for version 7.7 and 8.1 onwards."21:38
oneswigThat was the story in February21:38
trandlesah yeah, thx for the link21:39
trandlesI've heard of this blog post but hadn't read it myself21:39
*** jamesmcarthur has joined #openstack-meeting21:40
oneswigIt's somewhat biased :-)21:40
trandlesjust a tad21:40
martialWe can always let them know21:41
trandlesmartial: I haven't contacted Erwan yet but plan to21:41
trandlesthx for the introduction21:41
martialYou walked as I was talking about you:) small world21:42
*** jamesmcarthur has quit IRC21:44
oneswigjmlowe: that exploit on threadpost is not quite as bad as the whatsapp one...21:44
*** dklyle has quit IRC21:44
*** dklyle has joined #openstack-meeting21:44
*** bbowen has joined #openstack-meeting21:48
oneswigI think everyone here's on the Slack channel set up by martial - anyone want the sign-up link?21:48
trandlesbtw - ZombieLoad is scary21:49
trandlestl;dr disable hyperthreading21:50
*** dklyle has quit IRC21:50
trandlesZombieLoad can be triggered in virtual machines, which are meant to be isolated from other virtual systems and their host device.  Daniel Gruss, one of the researchers who discovered the latest round of chip flaws, said it works “just like” it does on PCs and can read data off the processor. That’s potentially a major problem in cloud environments where different customers’ virtual machines run on the same server h21:51
jandersoneswig: I think I will need to switch accounts - can I re-invite myself on a different email address?21:51
oneswigSure - martial is the admin, if you need super-powers :-)21:52
jandersok!21:52
jmloweugh, I hadn't read that carefully to see the exploitable in vms part21:52
trandlesthe abstract of the whitepaper (https://zombieloadattack.com/zombieload.pdf) says "We discuss both short and long-term mitigation approaches and arrive at the conclusion that disabling hyperthreading is the only possible workaround to prevent this extremely powerful attack on current processors."21:53
jmloweI saw Intel was rolling new microcode, so that's not sufficient?21:53
trandlesdunno21:54
*** mattw4 has quit IRC21:54
martialLet me get the link again21:55
martialhere https://join.slack.com/t/os-scientific-sig/shared_invite/enQtNjIyOTU1NjU3Njg1LTVjY2QzNTkyMjVmZjIzNjI2MDYzNjcxMzExMDY5NDQ0MTc1NGRjMzk2ZTE2N2VjZjJiMzlmMGM2MGJjZjY4YzA21:56
martialjanders, I think you can change your email from within the slack itself21:57
jandersok, thank you martial21:57
martialof note, Blair and Stig are also admin on the slack21:59
oneswigtrandles: If I read this right, a user process can access privileged memory, data from other VMs or the hypervisor, even SGX protected regions?21:59
martialand on that we are at the end of the hour21:59
jandersthanks guys!21:59
oneswigThanks all and good night :-)22:00
martialseems scary indeed22:00
oneswig#endmeeting22:00
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"22:00
openstackMeeting ended Tue May 14 22:00:18 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)22:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/scientific_sig/2019/scientific_sig.2019-05-14-21.00.html22:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/scientific_sig/2019/scientific_sig.2019-05-14-21.00.txt22:00
openstackLog:            http://eavesdrop.openstack.org/meetings/scientific_sig/2019/scientific_sig.2019-05-14-21.00.log.html22:00
*** rcernin has joined #openstack-meeting22:00
*** jmlowe has left #openstack-meeting22:00
*** trandles has quit IRC22:05
*** janders_ has quit IRC22:09
*** janders has quit IRC22:11
*** oneswig has quit IRC22:15
*** whoami-rajat has quit IRC22:18
*** armax has quit IRC22:26
*** enriquetaso has quit IRC22:32
*** yamamoto has joined #openstack-meeting22:40
*** yamamoto has quit IRC22:45
*** _erlon_ has quit IRC22:52
*** mattw4 has joined #openstack-meeting22:54
*** ttsiouts has joined #openstack-meeting22:58
*** hemna has quit IRC23:11
*** jamesmcarthur has joined #openstack-meeting23:17
*** hongbin has quit IRC23:20
*** carloss has quit IRC23:20
*** armax has joined #openstack-meeting23:25
*** yamamoto has joined #openstack-meeting23:26
*** ttsiouts has quit IRC23:32
*** macza has quit IRC23:33
*** rbudden has quit IRC23:40
*** irclogbot_1 has quit IRC23:45
*** lbragstad has quit IRC23:47
*** irclogbot_1 has joined #openstack-meeting23:48
*** jamesmcarthur has quit IRC23:49
*** jamesmcarthur has joined #openstack-meeting23:49
*** lseki has quit IRC23:51
*** jamesmcarthur has quit IRC23:54
*** njohnston has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!