Monday, 2018-08-06

*** frankwang has joined #openstack-release00:27
*** chason has joined #openstack-release01:21
*** hongbin has joined #openstack-release01:39
*** EmilienM has quit IRC02:39
*** EmilienM has joined #openstack-release02:40
*** udesale has joined #openstack-release03:52
*** ricolin has joined #openstack-release03:54
*** hongbin has quit IRC04:09
*** frankwang has quit IRC04:11
*** frankwang has joined #openstack-release04:12
*** ykarel has joined #openstack-release04:20
*** frankwang has quit IRC04:51
*** udesale has quit IRC05:34
*** frankwang has joined #openstack-release05:40
openstackgerritwanghao proposed openstack/releases master: Relase Zaqar RC-1 for Rocky  https://review.openstack.org/58902105:58
*** jtomasek has joined #openstack-release05:58
openstackgerritwanghao proposed openstack/releases master: Relase zaqar-ui RC-1 for Rocky  https://review.openstack.org/58902206:02
*** ifat_afek has joined #openstack-release06:06
*** pcaruana has joined #openstack-release06:07
*** udesale has joined #openstack-release06:11
*** egonzalez has joined #openstack-release06:40
*** amoralej|off is now known as amoralej07:09
*** ajo has joined #openstack-release07:09
*** jhesketh_ has joined #openstack-release07:10
*** jhesketh has quit IRC07:11
*** ykarel is now known as ykarel|lunch07:53
*** dtantsur|afk is now known as dtantsur07:59
*** olivierbourdon38 has joined #openstack-release09:00
*** ykarel|lunch is now known as ykarel09:03
*** olivierbourdon38 has quit IRC09:23
*** olivierbourdon38 has joined #openstack-release09:29
ttxnotmyname, smcginnis: I think the discrepancy in deadlines is due to the difference between the first release candidate and the final one09:35
ttxBasically for cycle-with-intermediary things we want a recent release and cut the stable branch from it around RC1 time09:35
ttxbut then you can release point releases off that stable branch up to the week before the coordinated release09:36
ttxthe "around RC1 time" has been a bit flexible in the past, rather than a strict deadline09:37
ttxas long as we know it's coming...09:38
*** frankwang has quit IRC10:15
*** ricolin has quit IRC10:31
*** udesale has quit IRC10:34
*** dave-mccowan has joined #openstack-release10:46
*** dave-mccowan has quit IRC11:03
*** rosmaita has joined #openstack-release12:03
*** radez has joined #openstack-release12:05
evrardjphello folks12:08
*** e0ne has joined #openstack-release12:08
evrardjpdhellmann: smcginnis I know it's a little late to start this work, but rather late than never: I will start to work on the change for openstack-ansible we discussed at the summit: having OSA roles in a separate file, and deal with those roles releasing as what we do in devstack: only care about branching. I will ping you if help is needed12:10
*** e0ne has quit IRC12:13
dhellmannevrardjp : is that for rocky, or for stein?12:24
smcginnisMight be safer at this point planning to do that right away once stein gets going.12:28
evrardjpI like to live dangerously?12:29
smcginnis:)12:29
evrardjpIt might be only for stein, but if it's possible to do it for rocky, I am fine with that. I doubt it will be possible though.12:30
smcginnisevrardjp: At least a great time to start looking at what needs to be done I think. Do let us know if you need help.12:32
evrardjpYes I figured it was too late when milestone 1 was already in, we were discussing during M2, so there is indeed no urgency.12:33
evrardjpbut right before branching seemed a good idea :D12:33
evrardjp(for my learning experience at least)12:33
evrardjpanyway... keep you in the loop !12:33
smcginnisGood luck@12:33
smcginniss/@/! :)12:33
*** ifat_afek has quit IRC12:34
*** dtantsur is now known as dtantsur|brb12:36
openstackgerritSpyros Trigazis proposed openstack/releases master: Release python-magnumclient 2.10.0  https://review.openstack.org/58913812:41
EmilienMrelease-managers: Happy Monday! please take a look at https://review.openstack.org/#/q/project:openstack/releases+owner:%22Emilien+Macchi+%253Cemilien%2540redhat.com%253E%22+status:open12:41
*** dave-mccowan has joined #openstack-release12:53
*** egonzalez has quit IRC13:02
*** dave-mccowan has quit IRC13:03
*** rosmaita has quit IRC13:10
*** edmondsw has joined #openstack-release13:13
openstackgerritJean-Philippe Evrard proposed openstack/releases master: Release OpenStack-Ansible queens/17.0.8  https://review.openstack.org/58914913:15
*** jroll has quit IRC13:19
*** jroll has joined #openstack-release13:19
*** rosmaita has joined #openstack-release13:23
openstackgerritJean-Philippe Evrard proposed openstack/releases master: Release OpenStack-Ansible pike/16.0.17  https://review.openstack.org/58915513:32
*** olivierbourdon38 has quit IRC13:38
*** mriedem has joined #openstack-release13:41
*** pkovar has joined #openstack-release13:41
*** olivierbourdon38 has joined #openstack-release13:41
*** lbragstad has joined #openstack-release13:44
*** olivierbourdon38 has quit IRC13:44
*** olivierbourdon38 has joined #openstack-release13:44
*** olivierbourdon38 has quit IRC13:46
*** olivierbourdon38 has joined #openstack-release13:47
*** pkovar has quit IRC13:47
*** olivierbourdon38 has quit IRC13:47
*** pkovar has joined #openstack-release13:51
*** clarkb has joined #openstack-release13:56
*** olivierbourdon38 has joined #openstack-release13:59
openstackgerritJean-Philippe Evrard proposed openstack/releases master: Release OpenStack-Ansible ocata/15.1.25  https://review.openstack.org/58917414:19
*** annabelleB has joined #openstack-release14:24
*** smcginnis[m] has quit IRC14:26
*** szaher has quit IRC14:26
*** dtantsur|brb has quit IRC14:26
*** dtantsur has joined #openstack-release14:28
*** e0ne has joined #openstack-release14:43
*** bobh has joined #openstack-release14:45
*** dave-mccowan has joined #openstack-release14:57
*** rpioso|afk is now known as rpioso15:00
*** pcaruana has quit IRC15:12
*** e0ne has quit IRC15:14
*** olivierbourdon38 has quit IRC15:24
*** ykarel has quit IRC15:28
*** AJaeger has joined #openstack-release15:31
AJaegerrelease team, can you EOL tripleo, please? See https://review.openstack.org/#/c/583856/ which has mwhahaha's +115:31
smcginnisAJaeger: Oh right, I've been meaning to check in with tonyb on that, but I will push that through.15:33
AJaegerthanks, smcginnis15:34
smcginnisNo problem15:34
*** ricolin has joined #openstack-release15:38
openstackgerritTakashi NATSUME proposed openstack/releases master: Fix OpenStack Summit Berlin schedule  https://review.openstack.org/58920315:49
openstackgerritMerged openstack/releases master: Tag final newton releases for tripleo deliverables  https://review.openstack.org/58385616:10
*** bobh has quit IRC16:18
openstackgerritMerged openstack/releases master: Fix OpenStack Summit Berlin schedule  https://review.openstack.org/58920316:25
openstackgerritMonty Taylor proposed openstack/releases master: Release 1.3.0 of os-service-types  https://review.openstack.org/58921916:28
dhellmannsmcginnis , ttx: do we want to start approving rc1 and branch requests, or wait until closer to the deadline?16:29
smcginnisI suppose we can start now if folks already think they are ready for it.16:29
ttxI'd say start now16:30
smcginnisMight be good to spread things out as much as possible so we aren't doing it all late in the week.16:30
ttxRC1 is expensive but rc2 is cheap16:30
dhellmannok, I'll do zaqar then16:31
dhellmannand solumn16:31
dhellmannttx: when you have a few minutes, I think we're waiting for your +2 on https://review.openstack.org/588604 and https://review.openstack.org/58860516:32
ttxlet me see16:32
dhellmannsmcginnis : we had 2 client release requests; I have asked them to send email for FFEs16:32
ttxall approved16:34
smcginnisOK, good.16:35
openstackgerritMerged openstack/releases master: Relase Zaqar RC-1 for Rocky  https://review.openstack.org/58902116:45
openstackgerritMerged openstack/releases master: Relase zaqar-ui RC-1 for Rocky  https://review.openstack.org/58902216:45
openstackgerritMerged openstack/releases master: Release solum 5.7.0 and cut stable/rocky  https://review.openstack.org/58887816:46
openstackgerritMerged openstack/releases master: add rocky-3 milestone tags for projects that missed  https://review.openstack.org/58860416:46
*** e0ne has joined #openstack-release16:47
*** evrardjp has quit IRC16:51
*** hongbin has joined #openstack-release16:52
*** pkovar has quit IRC16:54
*** bobh has joined #openstack-release16:55
*** olivierbourdon38 has joined #openstack-release16:56
openstackgerritMerged openstack/releases master: remove freezer and searchlight from rocky series  https://review.openstack.org/58860516:57
*** bobh has quit IRC17:02
*** bobh has joined #openstack-release17:02
*** amoralej is now known as amoralej|off17:03
*** bobh has quit IRC17:07
dhellmannsmcginnis , prometheanfire : FFE for python-magnumclient in http://lists.openstack.org/pipermail/openstack-dev/2018-August/132916.html17:16
*** ekcs has joined #openstack-release17:21
*** annabelleB has quit IRC17:28
*** tellesnobrega has joined #openstack-release17:34
prometheanfiredhellmann: replied (and re-subjected)17:36
tellesnobregasmcginnis, we backported all patches to sahara-extra stable/rocky, do I need to send a new release patch? if so, do I need to update version number, 9xx was used in queens, should it be changed to 10?17:37
dhellmannprometheanfire : thanks17:39
*** olivierbourdon38 has quit IRC17:40
*** annabelleB has joined #openstack-release17:40
*** evrardjp has joined #openstack-release17:41
*** bobh has joined #openstack-release17:48
*** dtantsur is now known as dtantsur|afk17:53
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/releases master: Releasing sahara-extra 9.2.0  https://review.openstack.org/58792617:53
*** pkovar has joined #openstack-release17:57
*** e0ne has quit IRC18:05
*** ricolin has quit IRC18:11
*** e0ne has joined #openstack-release18:14
*** bobh has quit IRC18:23
*** bobh has joined #openstack-release18:23
smcginnistellesnobrega: The branch is already created on there, so you don't want to update the patch with a new branch location.18:43
smcginnistellesnobrega: Other than that, I think it looks good.18:43
tellesnobregaok, thanks18:44
smcginnisdhellmann: Any idea what this is about? smcginnis@devvm1:~$ python test.py18:45
smcginnisHah, probably not that.18:45
smcginnishttp://logs.openstack.org/ad/ad52d8e0d84f68feee29a3dab19436a750db0c23/release-post/publish-static/c799ae1/job-output.txt.gz#_2018-08-06_17_13_03_74254418:45
tellesnobregasmcginnis, anything that I can do on that one?19:08
smcginnistellesnobrega: Nope, just the branching bit.19:15
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/releases master: Releasing sahara-extra 9.2.0  https://review.openstack.org/58792619:15
tellesnobregasmcginnis, that one is done19:15
smcginnistellesnobrega: Looks good. I expect that one should pass. Once we get the zuul results I'll send that through.19:16
smcginnisThanks!19:16
tellesnobregathanks :)19:16
openstackgerritSean McGinnis proposed openstack/releases master: Add stable/rocky branches for client libs  https://review.openstack.org/58723319:21
*** annabelleB has quit IRC19:29
*** annabelleB has joined #openstack-release19:34
openstackgerritMerged openstack/releases master: Releasing sahara-extra 9.2.0  https://review.openstack.org/58792619:37
*** pkovar has quit IRC19:58
*** e0ne has quit IRC20:11
dhellmannsmcginnis : i don't. maybe there's an issue with the server? I'd bring fungi or clarkb in on it20:18
dhellmannI hope we didn't just whack the web site20:18
fungilooking20:19
fungithe https://releases.openstack.org/ still has normal-looking content20:19
clarkband /srv/static/releases exists20:20
clarkbI think the source side is what did not exist20:20
clarkbnot thedest disde20:20
fungilooks like the errors are about missing source path not missing dest20:20
fungiyeah, that20:20
clarkbwhich could be the race we've had in the psat but now we do supercedent queueswhich should prevent that20:21
clarkbunless multiple projects can run release-post that updates the same dir20:21
fungie.g. no .doctrees/bexar/.~tmp~/index.doctree in the working directory of that task20:21
*** annabelleB has quit IRC20:21
fungilooks like it only errors up through kilo. are we maybe trying to upload content we're not building?20:23
dhellmannlet me build it locally and see what we have20:24
dhellmannwe should have all of those old releases now20:24
*** annabelleB has joined #openstack-release20:24
dhellmannsome of those files are general sphinx things, like genindex.html and index.html20:25
dhellmannsmcginnis : did more than one job fail with that error?20:25
dhellmannthose files definitely exist in the output I just built20:27
clarkbits specifically on the rename step which makes me think the copy went fine to the remote (static.o.o) then when it goes to do the atomic rename that failed because the source was gone20:29
clarkbwhich had happened with that pre supercedent post pipeline race20:29
clarkbit can still happen if we run the same job against multiple projects I think, but otherwise I'm confused20:29
smcginnisdhellmann: It was just the one that I saw.20:32
smcginnisWas wondering if it was our concurrent issue again, but I don't think anything else was running through at the same time.20:33
smcginnisCould have just been a quirk.20:33
* dhellmann doesn't like this sort of quirk20:33
smcginnisYeah20:33
dhellmannclarkb : by "same job" do you mean 2 things publishing to the static server at the same time, but out of different repos?20:34
dhellmannI wonder if we should put all of those jobs into the same queue20:34
clarkbdhellmann: yes, if the filenames overlap20:34
clarkbdhellmann: because rsync could copy the same tmp file in that case then the other job would find the file is gone20:34
dhellmannthe openstack-manuals repo would have some files with names like those20:34
clarkbs/copy/rename/20:34
smcginnisI thought a later release patch would catch things up, but Summit still shows wrong week - https://releases.openstack.org/stein/schedule.html20:34
dhellmannbut they go to different web servers20:34
dhellmannsmcginnis : maybe we should land a trivial change to force a rebuild?20:35
smcginnisdhellmann: I suppose. Have anything you've been meaning to update?20:41
dhellmannsmcginnis : nothing that's ready to go (or likely to be in < a week)20:41
smcginnisI'm sure I've got a typo somewhere I can correct.20:43
openstackgerritSean McGinnis proposed openstack/releases master: Trivial change to update docs  https://review.openstack.org/58928320:47
*** bobh has quit IRC20:55
smcginnisdhellmann: The Magnum client release was granted a FFE. Want to approve that since you have the -1 on there? Or at least remove the -1 so it doesn't look like you have an objection to it? :)20:55
smcginnishttps://review.openstack.org/#/c/589138/20:55
dhellmannon it20:56
dhellmannI sure hope that release works20:56
smcginnisFingers crossed20:57
dhellmannit's too late now, but I wonder if we want to have the release and branch happen in separate commits when there is only 1 release20:58
smcginnisWould there be a need to separate them?20:59
dhellmannwell, if something goes wrong with the release, creating a new one involves backporting patches21:02
dhellmannif we branch separately, then if the release fails they can fix it on master before branching21:02
dhellmannit's probably not a big deal, it's just a question of avoiding risk21:02
smcginnisWhen they are together, we don't branch if the release part fails I thought.21:02
dhellmannif the tag is applied but the package doesn't build we still get the branch21:03
dhellmannthe branch would only fail if the tag isn't there21:03
openstackgerritMerged openstack/releases master: Trivial change to update docs  https://review.openstack.org/58928321:03
smcginnisAh, true.21:03
openstackgerritMerged openstack/releases master: Release python-magnumclient 2.10.0  https://review.openstack.org/58913821:03
smcginnisI think usually the failures after that point have been from things outside of the repo. I'm sure there are exceptions though.21:03
smcginnisI could see deferring all stable branching until RC1 week+1 unless they know they need to start accepting patches for the next release cycle.21:04
smcginnisThat could eliminate some of the backporting during this window that we've had.21:04
dhellmannthat's an interesting idea21:05
dhellmannin the past teams have wanted to open master up to new stuff pretty quickly so we didn't want to wait21:05
dhellmannand because we have to coordinate it for most of the repos to happen close together, it was easier to just set the date21:05
smcginnisYeah, I know my experience was folks didn't want to slow down.21:06
dhellmanna lot of people are already working on stein21:06
* dhellmann is21:06
smcginnisDoc publishing was fine that time and the stein schedule reflects the updated date.21:14
*** edmondsw has quit IRC21:15
fungiclarkb: do we really not namespace those temporary directories by job name or anything?21:16
clarkbfungi: I don't think rsync does no21:16
clarkbits an rsync thing aiui21:16
fungiohh21:16
clarkbis that configurable?21:16
fungilooking... nearly everything about rsync is though21:16
fungi--temp-dir maybe?21:17
fungi"create temporary files in directory DIR"21:17
smcginnisSounds promising21:17
fungithe long description looks like that's exactly what it's for21:17
fungiclarkb: is the rsync there being called from a playbook i guess? not zuul itself?21:18
clarkbfungi: ya21:18
fungioh, by ansible internals though, not direct invocation21:19
fungihttp://logs.openstack.org/ad/ad52d8e0d84f68feee29a3dab19436a750db0c23/release-post/publish-static/c799ae1/ara-report/file/468db47e-a377-457e-8c1c-16a1aab52ad1/#line-1621:19
fungisynchronize has a parameter for passing rsync opts too though right?21:20
fungiaha, https://docs.ansible.com/ansible/latest/modules/synchronize_module.html says it's called rsync_opts21:21
*** harlowja has joined #openstack-release21:21
fungithough i find it extra strange that rsync utilizes a predictable temporary path on the remote system21:22
smcginnisSeems like a dangerous default behavior.21:23
fungiikr?21:24
fungioh, it's using --delay-updates according to http://logs.openstack.org/ad/ad52d8e0d84f68feee29a3dab19436a750db0c23/release-post/publish-static/c799ae1/ara-report/result/2b5f549d-92ed-4809-8e20-ec207368173f/21:25
fungiso per the manpage we may need to set --partial-dir instead of --temp-dir21:26
*** bobh has joined #openstack-release21:28
fungihrm... though the more i read this the more it looks like those .~tmp~ directories are being created relative to /srv/static/releases/ so shouldn't conflict with e.g. openstack-manuals because it would create its tempfiles under a different root?21:31
fungiaccording to --delay-updates description:21:32
fungiBy default the files are placed into a directory named  ".~tmp~" in  each  file’s  destination directory21:32
fungiso unless the manpage is wrong, i'm unconvinced this is the problem21:33
smcginnisHmm. It looks like it was a more generic path than each file's destination directory.21:34
clarkbfungi: if two jobs copy to the same destination was my concern21:34
clarkbfungi: we've seen that before for sure21:34
clarkbsupercedent should've reduced or prevented it though21:34
*** bobh has quit IRC21:35
smcginnisWe can see if we hit it again and then track down the exact conditions.21:35
fungiclarkb: yeah, the remaining concern seemed to be different jobs using similar filenames at the destination relative to their target dirs, but looks like if they have different target directories then the base path to those tempfiles will also be different21:36
openstackgerritSean McGinnis proposed openstack/releases master: Remove unused --etherpad opt from list_weeks  https://review.openstack.org/58929321:37
*** bobh has joined #openstack-release21:47
*** bobh has quit IRC22:02
*** bobh has joined #openstack-release22:03
*** annabelleB has quit IRC22:07
*** bobh has quit IRC22:07
*** annabelleB has joined #openstack-release23:07
*** bobh has joined #openstack-release23:12
*** hongbin has quit IRC23:19
*** rpioso is now known as rpioso|afk23:37
*** annabelleB has quit IRC23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!