Monday, 2018-01-22

ianwpabelanger: ok ... does that get it working enough for glean?00:01
ianwclarkb: as in in creates a broken env?00:01
ianwor creating the env is broken?00:02
*** yankcrime has quit IRC00:04
fungiseems like there was related discussion either in here or in #zuul about it00:04
*** _nick has joined #openstack-infra00:05
*** sree has joined #openstack-infra00:08
*** armax has quit IRC00:10
*** sree has quit IRC00:12
*** tosky has quit IRC00:16
*** dave-mccowan has joined #openstack-infra00:18
pabelangerianw: yah, issue was zuul-env was using python2 to install zuul (zuul-cloner)00:27
*** sree has joined #openstack-infra00:27
pabelangerso, elements need to pip install 2.6.000:27
pabelangeror stop using zuul-cloner, because we have ansible role for that now00:28
pabelanger(on DIB)00:28
pabelangerianw: I haven't tested glean / DIB on the nodepool patches. I can look at it in the morning00:28
*** sree has quit IRC00:32
*** reddhatt has joined #openstack-infra00:32
ianwpabelanger: ok, i'll see how i go, will need to get them fixed up00:34
reddhattHi all!00:34
reddhattI'm looking for some tips to install an Openstack environment.00:35
reddhattI tried the puppet manifests available for each module, in https://docs.openstack.org/puppet-openstack-guide/latest/contributor/module-list.html, and I could not succeed.00:35
reddhattIs there any manifest for puppet, or something for SaltStack or Ansible?00:36
reddhattThank you in advance.00:36
*** olaph1 has joined #openstack-infra00:41
ianwreddhatt: this probably isn't the channel, as we run the hosting infrastructure etc for the project.  from your nick, are you interested in a redhat environment?  if so, i'd suggest you start with https://www.rdoproject.org//00:41
reddhattOps.... I'm looking for Debian. My nick is based on historical preferences... kkkkkk00:42
*** olaph has quit IRC00:42
*** dingyichen has joined #openstack-infra00:45
*** edmondsw has joined #openstack-infra00:46
*** edmondsw has quit IRC00:51
*** benj_ has joined #openstack-infra00:54
reddhatt@ianw, thank you!00:55
*** abelur_ has joined #openstack-infra01:00
*** cuongnv has joined #openstack-infra01:08
*** dhill_ has quit IRC01:12
clarkbianw: the problem was that with feature/zuulv3 merged into master there is no more zuul-cloner and you need python3. So ya we want to install older zuul with python2 instead01:13
ianwclarkb: but zuul-cloner in the image is a private copy put in via fetch-zuul-cloner?01:18
ianw(apologies, i haven't read scrollback on this)01:18
clarkbianw: we still rely on the virtualenv to provide the python bits. We can probably actually stop installing zuul into the virtualenv and things would work? but we do need the virtualenv for pyyaml iirc01:19
ianwclarkb: hmm, i wasn't aware we were fully installing zuul into it; i'm talking about https://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/fetch-zuul-cloner/tasks/main.yaml01:20
clarkboh hrm then maybe we just haven't removed the necessary bits from the dib elements list?01:21
clarkbalso I never checked our logs to see if it was happening to us it was mrhillsman that it happened to on image build01:21
mrhillsmano/01:22
ianwclarkb mrhillsman : we removed the zuul-cloner element around the same time we got that job going01:22
ianwwe being pabelanger :)01:23
ianwhttps://review.openstack.org/#/c/522347/ being the review for that01:23
ianwso ... old elements?01:23
mrhillsmanah, i noticed that file was missing in new install01:27
*** aeng has quit IRC01:28
*** zhurong has joined #openstack-infra01:29
*** zhurong has quit IRC01:33
ianwmrhillsman: yeah, it was intentional :)  moving it into the ansible setup was the plan for when zuul-cloner went away.01:37
ianwwent away from the zuul source tree, i mean01:37
*** reddhatt has quit IRC01:42
*** markvoelker has joined #openstack-infra01:47
*** gongysh has joined #openstack-infra01:49
*** daidv has joined #openstack-infra01:51
*** hongbin has joined #openstack-infra02:02
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: builder: do not cleanup image for driver not managing image  https://review.openstack.org/53555202:06
*** rkukura has quit IRC02:09
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement a static driver for Nodepool  https://review.openstack.org/53555302:09
*** gcb has joined #openstack-infra02:09
openstackgerritIan Wienand proposed openstack-infra/system-config master: Add linaro cloud credentials  https://review.openstack.org/53618702:13
*** weshay has quit IRC02:14
*** weshay has joined #openstack-infra02:15
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Refactor run_handler to be generic  https://review.openstack.org/53555402:15
ianwthere's an unstaged perms change to a file in hieradata/production/group ; i'm going to assume it was corvus and commit because makes everything consistent02:17
*** kiennt26 has joined #openstack-infra02:19
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Refactor NodeLauncher to be generic  https://review.openstack.org/53555502:20
*** markvoelker has quit IRC02:21
*** zhurong has joined #openstack-infra02:32
*** liujiong has joined #openstack-infra02:33
mtreinishfungi: not that I have enough time to do the testing, but I was thinking more about a volumetric graph for the firehose data and we could do it to a limited degree by using QoS level as the 3rd axis02:39
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an OpenContainer driver  https://review.openstack.org/53555602:39
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement a Kubernetes driver  https://review.openstack.org/53555702:39
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an Amazon EC2 driver  https://review.openstack.org/53555802:39
mtreinishthere's still the data problem so we'd have to have a lot more data points on subscriber and publisher counts to make it a useful visualization02:40
*** annp has joined #openstack-infra02:50
*** yamamoto has joined #openstack-infra02:57
mtreinishfungi: also: https://github.com/mtreinish/firehose/commit/ee41a31e08bb52b9e0c59636c7dadac2f776d29f our data was kinda bogus. The updated graph makes it a bit clearer that we were basically just hititng bandwidth limits everywhere02:58
mtreinishwhich is why the only place with high throughput is in the corners02:58
*** rkukura has joined #openstack-infra03:05
*** liujiong has quit IRC03:17
*** markvoelker has joined #openstack-infra03:17
*** liujiong has joined #openstack-infra03:20
*** sree has joined #openstack-infra03:24
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] testing nodepool forcing to py3 in devstack job  https://review.openstack.org/53619803:37
ianwpabelanger: thanks, i think that's sufficient for the dib failures too, ^ will see03:37
ianwi'm not sure what it means for centos testing, though03:39
*** psachin has joined #openstack-infra03:39
ianwoh, i guess we never did anyway03:39
*** ramishra has joined #openstack-infra03:43
*** ramishra has quit IRC03:50
*** markvoelker has quit IRC03:51
*** ramishra has joined #openstack-infra03:52
*** links has joined #openstack-infra04:00
*** ekcs has joined #openstack-infra04:02
*** dave-mccowan has quit IRC04:21
*** abhishekk has joined #openstack-infra04:22
*** edmondsw has joined #openstack-infra04:22
abhishekkhi, I am experiencing TIME_OUT for python27 job on patch https://review.openstack.org/#/c/527635/04:22
abhishekkwhat is the way to get rid of this, can anyone suggest?04:23
*** coolsvap has joined #openstack-infra04:24
*** edmondsw has quit IRC04:27
*** ykarel|away has joined #openstack-infra04:34
*** bhavik1 has joined #openstack-infra04:37
*** zhurong has quit IRC04:39
*** pgadiya has joined #openstack-infra04:39
*** vivsoni has joined #openstack-infra04:46
*** gongysh has quit IRC04:47
*** markvoelker has joined #openstack-infra04:48
*** armaan has quit IRC04:56
*** armaan has joined #openstack-infra04:57
*** hongbin has quit IRC05:03
openstackgerritIan Wienand proposed openstack-infra/glean master: DNM: Testing nodepool jobs  https://review.openstack.org/53613505:06
*** bhavik1 has quit IRC05:07
ianwabhishekk: it looks like you've got legitimate failures in http://logs.openstack.org/35/527635/12/gate/openstack-tox-py27/8e8af04/job-output.txt.gz#_2018-01-20_23_11_27_12568005:08
ianwi.e. this doesn't seem to be related to the job infrastructure, but the job itself05:08
abhishekkianw, I am afraid as it is not failing in local environment05:09
ianwabhishekk: is it every job or just yours?  nothing is jumping out at me, but i'm no glance expert05:12
abhishekk ianw, not sure05:13
ianwwell that's always a good place to start, go through the latest runs in the project and see if it's something global05:17
AJaegerabhishekk: there must be some change in glance or the dependencies that introduces this, it fails for al glance changes but nowhere else. best discuss with glance team and debug...05:18
abhishekkAJaeger, ianw sure I will have a look05:18
AJaegerianw, frickler, could you review https://review.openstack.org/535875 and https://review.openstack.org/531339 , please?05:19
*** markvoelker has quit IRC05:21
*** gongysh has joined #openstack-infra05:26
prometheanfireAJaeger: talking about the glance27 fails?05:27
prometheanfirewe may be disabling it in reqs if it's not fixed soon, https://review.openstack.org/53608205:28
prometheanfirefreeze this week05:28
*** bobh has joined #openstack-infra05:33
*** ykarel|away is now known as ykarel05:34
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: zk: use kazoo retry facilities  https://review.openstack.org/53620905:38
*** stakeda has joined #openstack-infra05:40
*** threestrands has quit IRC05:40
*** zhurong has joined #openstack-infra05:45
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Handle os-api-ref in build-api-ref job  https://review.openstack.org/53587505:45
*** gongysh has quit IRC05:52
*** snapiri has quit IRC05:55
*** gongysh has joined #openstack-infra05:55
*** oanson has quit IRC05:55
*** yamahata has quit IRC05:56
AJaegerthanks, ianw05:59
AJaegerprometheanfire: yes, exactly that. If you disable it, I would remove from gate as well06:01
prometheanfireremove from gate?06:01
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/53621406:09
*** liujiong has quit IRC06:09
*** edmondsw has joined #openstack-infra06:11
*** armaan has quit IRC06:14
*** armaan has joined #openstack-infra06:14
*** edmondsw has quit IRC06:15
*** markvoelker has joined #openstack-infra06:18
openstackgerritMerged openstack-infra/nodepool master: Drop python2 virtualenv for devstack  https://review.openstack.org/53616806:22
*** bobh has quit IRC06:30
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: Do not call merger:cat when all config items are excluded  https://review.openstack.org/53613406:31
*** sree_ has joined #openstack-infra06:33
*** sree_ is now known as Guest8625006:34
*** sree has quit IRC06:36
*** makowals has joined #openstack-infra06:40
*** markvoelker has quit IRC06:51
*** spiffxp has quit IRC06:54
*** vipul has quit IRC06:54
*** oanson has joined #openstack-infra07:02
*** snapiri has joined #openstack-infra07:05
*** alexchadin has joined #openstack-infra07:05
AJaegerprometheanfire: do not run in gate pipelin, only run non-voting in change pipeline. Your change keeps it in gate pipeline07:13
*** rcernin has quit IRC07:17
prometheanfireAJaeger: ah07:25
*** andreas_s has joined #openstack-infra07:26
prometheanfiredone07:28
prometheanfire4 ssh connections to do git review :|07:28
*** Guest86250 has quit IRC07:32
*** sree has joined #openstack-infra07:32
prometheanfireta07:32
AJaegerLGTM07:33
openstackgerritMerged openstack-infra/project-config master: Use tempest-plugin-jobs project template for all tempest plugins  https://review.openstack.org/53133907:33
openstackgerritMerged openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/53621407:33
prometheanfireya, also noticed newton still being refrenced, that should go at this point too07:33
prometheanfirebut tomorrow07:33
AJaegerprometheanfire: https://review.openstack.org/#/c/536054 is ready for review (was waiting for 531339 to merge), if you have still have time , please07:38
AJaegerprometheanfire: there's still a newton branch for requirements - let's retire that first. tonyb has been coordinating this, might want to chat with him07:39
prometheanfiredone07:41
*** ykarel is now known as ykarel|lunch07:41
prometheanfireya, I'm hoping to talk to him today about freeze specifics07:41
AJaegerthanks07:41
*** slaweq has joined #openstack-infra07:47
*** markvoelker has joined #openstack-infra07:48
*** jtomasek has joined #openstack-infra07:49
*** pcaruana has joined #openstack-infra07:52
*** janki has joined #openstack-infra07:55
*** pcichy has joined #openstack-infra07:56
*** edmondsw has joined #openstack-infra07:59
tonybAJaeger: we can't retire it as it will break a bunch of existing users (as we found out when we did mitaka)08:02
AJaegerprometheanfire: ^08:03
AJaegertonyb: ;(08:03
*** edmondsw has quit IRC08:03
*** jtomasek has quit IRC08:04
AJaegertonyb: did we retire nova's mitaka branch in the meantime? Any other late comers?08:04
prometheanfiretonyb: which users?08:04
*** jtomasek has joined #openstack-infra08:04
*** florianf has joined #openstack-infra08:11
ajoanybody knows how to fix this https://review.openstack.org/#/c/535861/3     deliverables/pike/networking-ovn.yaml: openstack-infra/project-config/zuul.d/projects.yaml no release job specified for openstack/networking-ovn, one of ['publish-to-pypi'] needs to be included in ['system-required', 'openstack-python-jobs-neutron', 'publish-openstack-sphinx-docs', 'release-notes-jobs', 'translation-jobs',08:12
ajo'check-requirements', 'openstack-python35-jobs-neutron', 'periodic-jobs-with-neutron-lib-master', 'periodic-jobs-with-ovsdbapp-master', 'publish-to-pypi-neutron'] or no release will be published   ?08:12
prometheanfireanyway, nn all08:13
AJaegerajo: looks the test is wrong, you have publish-to-pypi-neutron there. Talk to release team...08:14
ajoThanks AJaeger: who is release team?08:14
*** alexchadin has quit IRC08:15
AJaegerajo: #openstack-release ;)08:15
ajo:D08:15
AJaegerajo: but just checking: You can change publish-to-pypi-neutron to publish-to-pypi, we don't need publish-to-pypi-neutron anymore08:16
*** amoralej|off is now known as amoralej08:16
AJaegerso, send a change to project-config to get rename the template08:17
AJaegeror discuss with release team first - and mordred as well08:17
ajoAJaeger: what template?, tbh, I'm a bit "monkey" here, just trying to get a release done, but I'm not very familiar with the release tools, I don't mind spending some time learning about it :D08:18
ajo(but not sure where to start) :)08:18
*** alexchadin has joined #openstack-infra08:18
ajo"not very familiar" == "completely unfamiliar" :D08:18
*** tesseract has joined #openstack-infra08:19
AJaegerajo, http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/projects.yaml#n991808:20
*** ykarel|lunch is now known as ykarel08:20
AJaegerjust change that single line and remove the "-neutron"08:20
AJaegerand ask release team whether that's the right thing, please. We should do it everywhere then...08:21
*** markvoelker has quit IRC08:21
ajoack, thanks08:23
*** ralonsoh has joined #openstack-infra08:26
openstackgerritMiguel Angel Ajo proposed openstack-infra/project-config master: DNM Change publish-to-pypi-neutron in networking-ovn  https://review.openstack.org/53624708:27
*** danpawlik has joined #openstack-infra08:29
*** slaweq_ has joined #openstack-infra08:31
*** dsariel has quit IRC08:34
*** efoley has joined #openstack-infra08:34
*** slaweq_ has quit IRC08:36
*** rossella_s has quit IRC08:36
*** pblaho has joined #openstack-infra08:37
*** oidgar has joined #openstack-infra08:37
*** rossella_s has joined #openstack-infra08:39
*** giblet is now known as gibi08:39
*** andreas_s has quit IRC08:43
*** andreas_s has joined #openstack-infra08:44
*** jpena|off is now known as jpena08:46
*** dingyichen has quit IRC08:46
*** migi has quit IRC08:49
AJaegerajo: why DNM? Remove that so that we can review and merge...08:49
openstackgerritVasyl Saienko proposed openstack-infra/project-config master: Add jobs for n-g-s tempest-plugin  https://review.openstack.org/53254308:50
*** migi has joined #openstack-infra08:50
*** slaweq_ has joined #openstack-infra08:53
*** andreas_s has quit IRC08:53
*** rossella_s has quit IRC08:53
*** shardy has joined #openstack-infra08:54
openstackgerritMiguel Angel Ajo proposed openstack-infra/project-config master: Change publish-to-pypi-neutron in networking-ovn  https://review.openstack.org/53624708:55
*** gaurangt has quit IRC08:56
*** gaurangt has joined #openstack-infra08:56
*** slaweq_ has quit IRC08:57
*** andreas_s has joined #openstack-infra08:58
*** alexchadin has quit IRC08:59
*** jpich has joined #openstack-infra08:59
*** shardy has quit IRC08:59
*** rossella_s has joined #openstack-infra08:59
*** shardy has joined #openstack-infra09:00
ajoAJaeger: ack, I removed, but still, it's not working it seems: https://review.openstack.org/#/c/535861/409:05
ajo(if it's taking the depends-on, of course)09:05
*** efoley has quit IRC09:06
*** rossella_s has quit IRC09:07
*** andreas_s has quit IRC09:07
*** andreas_s has joined #openstack-infra09:09
*** b_bezak has joined #openstack-infra09:09
*** jaosorior has joined #openstack-infra09:10
AJaegerajo: they might check the version in-tree and ignore depends-on. Not sure. But talk with the release team and then we can merge this. LEt me +2 based on my reading...09:10
*** rossella_s has joined #openstack-infra09:11
AJaegerfrickler: my point with the designate rename: IF you don't do it now, you use the old job name in several repos and then it becomes complicated (three changes for a rename at least). designate is nice since it uses a template, so easy to rename...09:12
*** andreas_s has quit IRC09:16
*** andreas_s has joined #openstack-infra09:17
AJaegerfrickler: could you put https://review.openstack.org/#/c/535719/ and https://review.openstack.org/#/c/513317/ and https://review.openstack.org/531058 on your review queue, please?09:17
*** dbecker has joined #openstack-infra09:19
*** markvoelker has joined #openstack-infra09:19
*** markvoelker has quit IRC09:23
*** derekh has joined #openstack-infra09:25
*** _nick is now known as yankcrime09:31
*** b_bezak has quit IRC09:32
*** gcb has quit IRC09:32
*** sshnaidm|off has quit IRC09:33
*** edmondsw has joined #openstack-infra09:47
*** kjackal has joined #openstack-infra09:47
*** bhagyashris has joined #openstack-infra09:49
*** edmondsw has quit IRC09:52
*** hashar has joined #openstack-infra09:53
*** dsariel has joined #openstack-infra09:55
*** kjackal has quit IRC09:55
*** e0ne has joined #openstack-infra09:55
*** yamamoto has quit IRC09:57
*** yamamoto has joined #openstack-infra09:58
*** kjackal has joined #openstack-infra09:58
*** pgadiya has quit IRC09:58
*** pgadiya has joined #openstack-infra09:59
*** b_bezak has joined #openstack-infra10:00
*** szaher has joined #openstack-infra10:02
*** yamamoto has quit IRC10:03
*** dtantsur|afk is now known as dtantsur10:05
*** sree has quit IRC10:06
*** sree has joined #openstack-infra10:06
*** kjackal has quit IRC10:07
*** yamamoto has joined #openstack-infra10:09
*** yamamoto has quit IRC10:10
*** sree has quit IRC10:11
*** yamamoto has joined #openstack-infra10:12
*** alexchadin has joined #openstack-infra10:16
*** yolanda has joined #openstack-infra10:17
*** tosky has joined #openstack-infra10:17
*** cshastri has joined #openstack-infra10:19
*** cuongnv has quit IRC10:21
*** e0ne has quit IRC10:21
*** electrofelix has joined #openstack-infra10:23
*** e0ne has joined #openstack-infra10:25
*** ldnunes has joined #openstack-infra10:27
openstackgerritlei zhang proposed openstack-infra/project-config master: Add api-ref jobs for Workflow service (mistral)  https://review.openstack.org/53629410:28
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Clean held nodes automatically after configurable timeout  https://review.openstack.org/53629510:28
*** pbourke has quit IRC10:32
*** zhurong has quit IRC10:33
*** pbourke has joined #openstack-infra10:33
*** d0ugal has quit IRC10:37
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Refactor status functions, add web endpoints, allow params This patch refactors status functions so that instead of having one function per output format, the output format is simply a parameter.  https://review.openstack.org/53630110:44
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Add a separate module for node management commands  https://review.openstack.org/53630310:47
*** zhurong has joined #openstack-infra10:51
*** tpsilva has joined #openstack-infra10:56
*** d0ugal has joined #openstack-infra10:57
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: webapp: add optional admin endpoint  https://review.openstack.org/53631910:59
*** abhishekk has quit IRC11:02
*** andreas_s has quit IRC11:07
*** andreas_s has joined #openstack-infra11:08
*** andreas_s has quit IRC11:12
*** vivsoni has quit IRC11:13
*** vivsoni has joined #openstack-infra11:14
*** ganso has joined #openstack-infra11:16
*** rfolco is now known as rfolco|rover11:16
*** cshastri has quit IRC11:16
*** yolanda has quit IRC11:17
*** andreas_s has joined #openstack-infra11:17
*** markvoelker has joined #openstack-infra11:19
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Restore branch conditional on tempest-full  https://review.openstack.org/53105811:20
*** annp has quit IRC11:22
openstackgerritMerged openstack-infra/project-config master: Cerberus/Sticks retirement  https://review.openstack.org/53571911:22
*** links has quit IRC11:23
*** gaurangt has quit IRC11:24
*** gaurangt has joined #openstack-infra11:25
*** hashar has quit IRC11:27
*** dizquierdo has joined #openstack-infra11:27
*** alexchadin has quit IRC11:28
*** gongysh has quit IRC11:30
*** bbc1__ has joined #openstack-infra11:31
*** bbc1__ has quit IRC11:31
*** andreas_s has quit IRC11:32
*** andreas_s has joined #openstack-infra11:33
*** openstackgerrit has quit IRC11:33
*** alexchadin has joined #openstack-infra11:33
*** links has joined #openstack-infra11:36
*** andreas_s has quit IRC11:42
*** janki has quit IRC11:43
*** wolverineav has joined #openstack-infra11:44
*** jkilpatr has joined #openstack-infra11:47
*** andreas_s has joined #openstack-infra11:47
*** sree has joined #openstack-infra11:49
*** andreas_s has quit IRC11:49
*** andreas_s has joined #openstack-infra11:50
*** rkukura has quit IRC11:50
*** wolverineav has quit IRC11:52
*** wolverineav has joined #openstack-infra11:53
*** psachin has quit IRC11:53
smcginnisWe had discussed an account that appears to be trying to game their numbers by +1ing patches after they have been approved a while back.11:53
smcginnisSomeone had tried contacting them via their listed emails to no luck.11:53
*** sree has quit IRC11:53
smcginnisI've now left several comments for them on some of the reviews asking them to stop.11:53
smcginnisCan we just disable this account?11:53
*** e0ne has quit IRC11:54
*** markvoelker has quit IRC11:54
*** tesseract-RH has joined #openstack-infra11:55
*** jkilpatr has quit IRC11:55
*** tesseract has quit IRC11:57
*** sshnaidm has joined #openstack-infra11:58
*** dsariel has quit IRC11:59
AJaegersmcginnis: I contacted two of them - one email bounced, one no reaction12:08
*** jkilpatr has joined #openstack-infra12:09
*** janki has joined #openstack-infra12:10
smcginnisAJaeger: I've left 3-4 comments on patches where they have done this, but they have still continued at least as of a few hours ago.12:11
fricklersmcginnis: based on previous discussion I think I could just set them to inactive in gerrit, can you point me to a sample review?12:11
smcginnisfrickler: Last one here: https://review.openstack.org/#/c/535984/12:12
*** openstackgerrit has joined #openstack-infra12:12
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Remove legacy jobs in networking-calico  https://review.openstack.org/51331712:12
*** dave-mccowan has joined #openstack-infra12:12
AJaegersmcginnis: yeah, that was the user that I got a "550 User suspended:" from the mail provider from12:14
fricklersmcginnis: AJaeger: set to inactive in gerrit, please let me know when you see more of this12:15
smcginnisAJaeger: I think I've seen the other account you are probably referring to, but this one from that patch has been the worse of the two from the ones I've seen.12:15
smcginnisfrickler: Thanks!12:15
AJaegerfrickler: best to #status log this to inform rest of team12:15
frickler#status log gerrit account 26576 has been set to inactive due to continued review spam12:16
openstackstatusfrickler: finished logging12:16
fricklerAJaeger: I was already typing this ;)12:16
AJaegerfrickler: we haven't done this before, so might be something for tomorrow's team meeting...12:16
bhagyashrishi all, Looking into the glance patches there is TIME_OUT for python27 job for exmple: https://review.openstack.org/#/c/527635/ what is the way to get rid of this, can anyone suggest?12:17
*** krtaylor has joined #openstack-infra12:19
AJaegerbhagyashris: looks like a glance specific problem, please ask that team...12:21
*** dave-mcc_ has joined #openstack-infra12:21
*** dave-mccowan has quit IRC12:22
bhagyashrisAJaeger: Hi, actually I have checked this specific for glance and looks that this is not failing in local environment. i will also discussed this with glance team12:23
*** claudiub has joined #openstack-infra12:25
*** adarazs is now known as adarazs_lunch12:26
openstackgerritMerged openstack-infra/project-config master: Remove legacy jobs in networking-calico  https://review.openstack.org/51331712:26
*** jpena is now known as jpena|lunch12:33
*** smatzek has joined #openstack-infra12:33
*** alexchadin has quit IRC12:35
*** cshastri has joined #openstack-infra12:35
*** oidgar has quit IRC12:40
*** alexchadin has joined #openstack-infra12:42
*** smatzek has left #openstack-infra12:44
*** kjackal has joined #openstack-infra12:47
*** markvoelker has joined #openstack-infra12:51
*** kjackal has quit IRC12:57
openstackgerritPaul Bourke (pbourke) proposed openstack-infra/bindep master: Add OracleLinux support  https://review.openstack.org/53635512:58
*** yamamoto has quit IRC12:59
*** markvoelker has quit IRC13:02
*** markvoelker has joined #openstack-infra13:02
*** kjackal has joined #openstack-infra13:05
openstackgerritBernard Cafarelli proposed openstack-infra/release-tools master: Add StoryBoard filter and tag tools  https://review.openstack.org/53480813:11
*** kjackal has quit IRC13:11
*** e0ne has joined #openstack-infra13:18
*** yamamoto has joined #openstack-infra13:19
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Clean held nodes automatically after configurable timeout  https://review.openstack.org/53629513:21
openstackgerritFabien Boucher proposed openstack-infra/zuul master: Tenant config dynamic loading of project from external script  https://review.openstack.org/53587813:21
*** jcoufal has joined #openstack-infra13:25
*** bobh has joined #openstack-infra13:25
*** trown|outtypewww is now known as trown13:26
*** jpena|lunch is now known as jpena13:26
*** trown is now known as trown|ruck13:26
*** gongysh has joined #openstack-infra13:27
*** adarazs_lunch is now known as adarazs13:27
*** cshastri has quit IRC13:28
*** zhurong has quit IRC13:28
*** kjackal has joined #openstack-infra13:30
*** rlandy has joined #openstack-infra13:31
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Refactor status functions, add web endpoints, allow params  https://review.openstack.org/53630113:32
*** bobh has quit IRC13:34
*** rosmaita has joined #openstack-infra13:35
*** andreas_s has quit IRC13:35
*** swest has quit IRC13:35
*** andreas_s has joined #openstack-infra13:35
*** amoralej is now known as amoralej|lunch13:38
*** dprince has joined #openstack-infra13:38
*** gcb has joined #openstack-infra13:39
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Clean held nodes automatically after configurable timeout  https://review.openstack.org/53629513:39
*** coolsvap has quit IRC13:41
*** pgadiya has quit IRC13:42
*** andreas_s has quit IRC13:43
*** andreas_s has joined #openstack-infra13:44
*** edmondsw_ has joined #openstack-infra13:46
*** kgiusti has joined #openstack-infra13:49
openstackgerritClint 'SpamapS' Byrum proposed openstack/gertty master: Py3 compat changes for diff view  https://review.openstack.org/53637013:51
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Refactor status functions, add web endpoints, allow params  https://review.openstack.org/53630113:53
*** dsariel has joined #openstack-infra13:54
*** swest has joined #openstack-infra13:54
*** esberglu has joined #openstack-infra13:55
*** b_bezak has quit IRC13:56
*** ykarel is now known as ykarel|afk13:56
*** edmondsw_ is now known as edmondsw13:56
*** b_bezak has joined #openstack-infra14:00
*** weshay is now known as weshay|rover14:00
*** ykarel|afk has quit IRC14:03
*** sree has joined #openstack-infra14:06
*** kopecmartin has joined #openstack-infra14:07
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Clean held nodes automatically after configurable timeout  https://review.openstack.org/53629514:08
*** finucannot is now known as stephenfin14:08
*** kopecmartin has quit IRC14:12
*** kopecmartin has joined #openstack-infra14:12
*** yamamoto has quit IRC14:17
*** yamamoto has joined #openstack-infra14:18
*** amoralej|lunch is now known as amoralej14:20
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Revert "Register term_handler for all zuul apps"  https://review.openstack.org/53637614:20
*** yamamoto has quit IRC14:23
rosmaitahello infra people ... i need help troubleshooting a gate issue which has caused the requirements team to remove glance py27 tests from their gate (which i am not convinced is a great idea, but that's a different issue).  tests pass locally, were passing last week, not sure what has happened.  here is a test patch: https://review.openstack.org/#/c/536070/14:24
*** alexchadin has quit IRC14:25
rosmaitahow can i determine the last time the openstack-tox-py27 gate ran successfully on a glance patch?14:25
*** dsariel has quit IRC14:25
*** eharney has joined #openstack-infra14:26
mnaserrosmaita: http://zuul.openstack.org/builds.html?job_name=openstack-tox-py27&project=openstack%2Fglance :)14:26
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: added get metadata promo codes endpoint  https://review.openstack.org/53637914:27
*** mriedem has joined #openstack-infra14:27
*** makowals has quit IRC14:27
rosmaitamnaser ty14:28
*** kien-ha has joined #openstack-infra14:28
*** b_bezak has quit IRC14:28
*** zhurong has joined #openstack-infra14:28
mnasernp14:28
*** Goneri has joined #openstack-infra14:28
*** Guest86649 is now known as dansmith14:29
*** ykarel|afk has joined #openstack-infra14:29
*** makowals has joined #openstack-infra14:30
*** dhill_ has joined #openstack-infra14:32
openstackgerritMerged openstack-infra/openstackid-resources master: added get metadata promo codes endpoint  https://review.openstack.org/53637914:32
dprincederekh, dtantsur: hi. Using the latest docker/services/ironic-* templates. I'm hitting a new permissions error on /var/lib/ironic. Seems that something has changed... Ironic conductor doesn't have perms to make 'master_images' dir on the fly anymore. I've got a work around (changing perms on /var/lib/ironic in the container) but wondering if you've seen it?14:34
* dprince sigh, wrong channel.14:34
dprincesorry!14:34
*** lucasagomes is now known as lucas-hungry14:34
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Add a separate module for node management commands  https://review.openstack.org/53630314:34
*** olaph has joined #openstack-infra14:41
*** olaph1 has quit IRC14:41
*** kjackal has quit IRC14:42
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: webapp: add optional admin endpoint  https://review.openstack.org/53631914:43
*** makowals has quit IRC14:44
*** kjackal has joined #openstack-infra14:45
rosmaitahello infra folks ... anyone aware of any image changes or gate updates shortly before 2018-01-20T06:02:29 on saturday?14:47
*** makowals has joined #openstack-infra14:47
*** yamamoto has joined #openstack-infra14:49
pabelangerianw: great!14:51
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul master: Use re2 for change_matcher  https://review.openstack.org/53638914:53
*** yamamoto has quit IRC14:54
*** gongysh has quit IRC14:54
corvusmordred: ah i think the master/devel problem is similar to the problem Shrews had with the nodepool devstack job.  let me collect my thoughts on that and write it up.14:55
mordredcorvus: awesome14:58
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul master: Slack driver  https://review.openstack.org/53639115:00
*** ihrachys has joined #openstack-infra15:04
*** rkukura has joined #openstack-infra15:05
*** dizquierdo has quit IRC15:05
*** armax has joined #openstack-infra15:06
Shrewsah, the fallback branch search thing15:08
*** r-daneel has joined #openstack-infra15:09
*** kjackal has quit IRC15:09
mordredcorvus: for the other thing, the traceback I was seeing, I put up https://review.openstack.org/#/c/536151/ - which feels more like a band-aid than the right fix15:09
corvusmordred: is the same repo involved?15:11
corvusmordred: maybe it's corrupted -- maybe something went bad on a migration or something...15:11
*** kien-ha has quit IRC15:12
corvusmordred: part of me wants to merge the change you proposed, but part of me worries that if the issue is something like filesystem corruption, it will mask it, and with no way to make it visible, the system as a whole will just gradually accumulate errors like that and get worse without us noticing, until someone finally notices every job retries 3 times before running...15:13
mordredcorvus: yah - I get the retry_limit error a LOT on the stack ending here: https://review.openstack.org/#/c/53586015:13
*** anticw has left #openstack-infra15:13
mordredcorvus: agree - that's my concern with that patch - I'm not sure how we discover deeper or more systemic issues with it in place15:13
pabelangerI'm going to reduce the TTL for nl01.o.o DNS to 5mins, and prepare to replace the 8GB instances with 2GB version15:14
corvusmordred: ok, i think we should at least understand what the error is in this case... can collect the errors so we can look for a pattern?15:14
*** dsariel has joined #openstack-infra15:15
mordredcorvus: sure! lemme make a list. https://review.openstack.org/#/c/533824/ and https://review.openstack.org/#/c/533825 both have retry-limit failures in their most recent build15:15
*** kjackal has joined #openstack-infra15:15
*** yamamoto has joined #openstack-infra15:15
*** yamamoto has quit IRC15:15
mordredcorvus: and https://review.openstack.org/#/c/533825 had them prior to the most recent recheck (just as the first three instances) ... but I'll make a list15:16
*** agopi has quit IRC15:16
rosmaitamordred are you aware of any changes to the gate around 06:30 on saturday? glance is seeing py27 test failures, but everything passes locally, i suspect an environmental problem: http://zuul.openstack.org/builds.html?job_name=openstack-tox-py27&project=openstack%2Fglance15:16
mordredcorvus: we don't get a build_id reported for those - shoudl I just collect change,revision+job?15:16
mordredrosmaita: not aware -but looking real quick15:17
mordredrosmaita: on first look, things like http://logs.openstack.org/37/520337/18/check/openstack-tox-py27/29bb721/job-output.txt.gz#_2018-01-22_11_04_07_71057015:18
mordredrosmaita: make me wonder if maybe a glance requirement changed15:18
*** sree_ has joined #openstack-infra15:18
mordredrosmaita: oh - are you running functional tests in that job on purpose?15:18
*** sree_ is now known as Guest876115:18
rosmaitayeah15:18
*** dsariel_ has joined #openstack-infra15:19
mordredkk15:19
corvusmordred: yeah.  there's a short build-id in the log url, so we get it at least for the last run, but we'll get 2 more failures if we collect it by change/revision/job15:19
*** armax has quit IRC15:20
mordredrosmaita: oh! oslo-test updated in global requirements at the end of last week15:20
mordrednm. glance isn't using that ... looking still15:21
*** armax has joined #openstack-infra15:21
*** hongbin has joined #openstack-infra15:22
*** sree has quit IRC15:22
rosmaitayeah, last requirements change we merged was on 1-17, and jobs ran successfully on 1-1815:22
mordredrosmaita: yah - but we apply constraints at job run time, so a constraints shift can affect you15:23
rosmaitagood point15:23
mordredkwellrosmaita: http://logs.openstack.org/37/520337/18/check/openstack-tox-py27/29bb721/tox/py27-5.log is what was installed in tox for a failure and http://logs.openstack.org/35/527635/12/check/openstack-tox-py27/e143008/tox/py27-5.log is the most recent success15:23
*** lucas-hungry is now known as lucasagomes15:23
*** rkukura has quit IRC15:24
mat128dmsimard: if you offer still stands, I'd like you to review https://review.openstack.org/#/c/531607/ and https://review.openstack.org/#/c/531609/ :)15:24
*** rkukura has joined #openstack-infra15:25
mordredrosmaita: doc8 and restructuredtext-lint are the only things that changed15:25
rosmaitayeah, and i can't see either of those being related to this problem15:26
*** trown|ruck is now known as trown|brb15:28
*** Goneri has quit IRC15:29
hrwhttps://review.openstack.org/#/c/536187/ - one step closer to AArch64 CI :D15:30
*** sshnaidm is now known as sshnaidm|bbl15:30
mordredrosmaita: I can't see any patches to zuul-jobs or openstack-zuul-jobs that would have had any impact15:31
*** rkukura has quit IRC15:31
rosmaitaarrrrrgh15:32
mordredrosmaita: so I'm down to suspecting something in a distro package dep changing -15:32
*** rkukura has joined #openstack-infra15:32
rosmaitayuck15:32
mordredwhich makes me want to add a general 'grab list of packages installed' at the beginning and end of each job like we do for devstack but for all jobs15:32
rosmaita++15:32
mordredand also makes me want to be able to get a build log without timestamps in it for more ease with diffing15:33
*** agopi has joined #openstack-infra15:33
*** vhosakot has joined #openstack-infra15:35
pabelangerokay, with DNS updates, I am going to stop nodepool-launcher on nl01.o.o and launch replacement instance (2GB)15:35
smcginnisIs there some way to see when new images were picked up? That might help knowing if it's a newer image with newer distro packages.15:35
pabelangersmcginnis: yes, we should be collecting when the image was build in logs. I think in zuul-info folder15:36
*** trown|brb is now known as trown|ruck15:36
pabelanger/etc/dib-builddate.txt is the file15:36
*** agopi_ has joined #openstack-infra15:36
smcginnispabelanger: We're trying to see what's going on with this job: http://logs.openstack.org/37/520337/18/check/openstack-tox-py27/29bb721/15:38
smcginnisDoesn't look like it captures those logs.15:38
mordredrosmaita: fwiw, here's a diff between a fail log and a success log with the timestamps stripped15:39
mordredrosmaita: https://gist.github.com/8f06493a270b4006fce35062d15d69df15:39
pabelangersmcginnis: here is the info: http://logs.openstack.org/37/520337/18/check/openstack-tox-py27/29bb721/zuul-info/zuul-info.ubuntu-xenial.txt15:39
*** agopi has quit IRC15:39
*** sshnaidm|bbl has quit IRC15:39
smcginnispabelanger: Ah, there it is - thanks!15:39
rosmaitalooks like last success was with image builddate 2018-01-17 21:5315:39
*** sshnaidm|bbl has joined #openstack-infra15:40
*** SergeyLukjanov has quit IRC15:40
*** ameade has quit IRC15:40
*** eglute has quit IRC15:40
*** mrmartin has quit IRC15:40
*** jklare has quit IRC15:40
*** jungleboyj has quit IRC15:40
*** mgkwill has quit IRC15:40
*** ilpianista_ has quit IRC15:40
*** portdirect has quit IRC15:40
*** DuncanT has quit IRC15:40
*** zaro has quit IRC15:40
*** mrhillsman has quit IRC15:40
*** edwarnicke has quit IRC15:40
*** kmARC has quit IRC15:40
*** mancdaz has quit IRC15:40
*** Adri2000 has quit IRC15:40
*** niedbalski has quit IRC15:40
*** igormarnat has quit IRC15:40
*** ekhugen_alt has quit IRC15:40
*** peterlisak has quit IRC15:40
*** igormarnat_ has joined #openstack-infra15:40
*** eglute_s has joined #openstack-infra15:40
*** peterlisak has joined #openstack-infra15:40
*** ekhugen_alt has joined #openstack-infra15:40
*** Adri2000 has joined #openstack-infra15:40
*** Adri2000 has quit IRC15:40
*** Adri2000 has joined #openstack-infra15:40
*** SergeyLukjanov has joined #openstack-infra15:40
*** mancdaz_ has joined #openstack-infra15:40
*** niedbalski_ has joined #openstack-infra15:40
*** igormarnat_ is now known as igormarnat15:41
*** mancdaz_ is now known as mancdaz15:41
*** jungleboyj has joined #openstack-infra15:41
*** zaro has joined #openstack-infra15:41
*** mgkwill has joined #openstack-infra15:41
*** mrmartin has joined #openstack-infra15:41
*** jklare has joined #openstack-infra15:41
*** kmARC has joined #openstack-infra15:41
*** portdirect has joined #openstack-infra15:41
*** edwarnicke has joined #openstack-infra15:41
*** mrhillsman has joined #openstack-infra15:41
*** ameade has joined #openstack-infra15:41
*** ramishra has quit IRC15:41
*** DuncanT has joined #openstack-infra15:41
*** panda is now known as panda|break15:42
*** sshnaidm|bbl has quit IRC15:42
*** bandini has quit IRC15:43
*** caphrim007 has quit IRC15:44
pabelangerI don't think anything changed on our side during that window. EG: new dib release or elements updates, but haven't confirmed15:44
*** caphrim007 has joined #openstack-infra15:44
*** Goneri has joined #openstack-infra15:44
*** bandini has joined #openstack-infra15:45
mordredrosmaita: I'm gonna make you a patch real quick to do a thing with your server logs on failure ...15:45
rosmaitaok15:45
*** ilpianista_ has joined #openstack-infra15:46
corvusmordred, Shrews: sent mail to zuul-discuss about the branch issue15:46
*** dsariel has quit IRC15:47
*** dsariel_ has quit IRC15:47
*** jamesmcarthur has joined #openstack-infra15:48
*** pkovar has joined #openstack-infra15:49
*** felipemonteiro_ has joined #openstack-infra15:49
*** caphrim007 has quit IRC15:49
pkovarhi, could an infra-root help the docs team delete these leftover files from afs, please?15:50
pkovarhttps://docs.openstack.org/draft/15:50
*** dsariel has joined #openstack-infra15:50
*** dsariel_ has joined #openstack-infra15:50
pkovarall of draft/15:50
pkovarnot needed anymore15:50
*** slaweq has quit IRC15:50
*** david-lyle has joined #openstack-infra15:53
*** niedbalski_ is now known as niedbalski15:54
*** niedbalski has joined #openstack-infra15:54
*** felipemonteiro__ has joined #openstack-infra15:55
*** derekh has quit IRC15:56
*** derekh has joined #openstack-infra15:56
*** ijw has joined #openstack-infra15:57
*** rkukura has quit IRC15:58
*** felipemonteiro_ has quit IRC15:58
*** shardy has quit IRC16:00
*** makowals has quit IRC16:02
*** shardy has joined #openstack-infra16:02
*** rkukura has joined #openstack-infra16:05
pabelangerokay, replacement nl01.o.o online, just waiting for DNS updates to be made on firewalls16:05
*** ykarel|afk is now known as ykarel16:05
snapiriInfra: Is there any issue with zuul? somehow it seems it is not reachable...16:06
snapiriI mean the zuul web-interface16:07
pabelangersnapiri: which URL?16:09
pabelangerhttp://zuul.openstack.org/ should be working16:09
pabelangerpossible you are using zuulv3.o.o16:10
pabelangerwhich was removed latest week16:10
snapiripabelanger: http://zuulv3.openstack.org/16:10
snapirioops :)16:10
pabelangersnapiri: ah, yes. That was just a temp URL, we've moved back to zuul.o.o now16:10
snapiriExcellent. Thanks16:10
pabelangernp!16:10
*** kopecmartin has quit IRC16:12
*** kopecmartin has joined #openstack-infra16:14
Shrewscorvus: replied! twice even.16:14
*** myoung is now known as myoung|biab16:15
corvusShrews: yeah, good point.  if we did need #2, it's likely for that kind of scenario...16:15
*** yamamoto has joined #openstack-infra16:16
*** david-lyle has quit IRC16:19
*** armax has quit IRC16:19
*** Guest8761 has quit IRC16:20
*** rkukura has quit IRC16:20
*** sree has joined #openstack-infra16:20
pabelangernew nl01.o.o (2GB) is running now16:21
*** rkukura has joined #openstack-infra16:21
*** armax has joined #openstack-infra16:22
*** sshnaidm|bbl has joined #openstack-infra16:23
Shrewspabelanger: \o/16:23
*** sshnaidm|bbl is now known as sshnaidm16:23
pabelangerI'm going to start on bringing nl04.o.o online16:24
*** sree has quit IRC16:25
*** panda|break is now known as [amda16:25
pabelangermordred: Shrews: corvus: mind a review of https://review.openstack.org/536035/ to stop providers on nl02 that we are moving to nl04.o.o16:25
*** [amda is now known as panda16:26
Shrewspabelanger: +3d16:26
*** yamamoto has quit IRC16:27
*** eharney has quit IRC16:27
pabelangerdanke!16:28
*** bobh has joined #openstack-infra16:28
*** david-lyle has joined #openstack-infra16:28
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul master: Use re2 for change_matcher  https://review.openstack.org/53638916:31
*** zhurong has quit IRC16:31
*** sree has joined #openstack-infra16:33
*** rkukura has quit IRC16:33
openstackgerritMerged openstack-infra/project-config master: Create nl04.o.o configuration  https://review.openstack.org/53603516:34
*** jamesmcarthur has quit IRC16:35
*** kopecmartin has quit IRC16:36
*** bhavik1 has joined #openstack-infra16:37
*** andreas_s has quit IRC16:39
*** kjackal has quit IRC16:42
*** bhavik1 has quit IRC16:42
*** kjackal has joined #openstack-infra16:43
*** andreas_s has joined #openstack-infra16:44
*** ykarel is now known as ykarel|afk16:44
*** kien-ha has joined #openstack-infra16:45
*** jpena is now known as jpena|brb16:46
pabelangerbringing nl04.o.o online16:48
*** tesseract-RH has quit IRC16:51
*** andreas_s has quit IRC16:53
*** sree has quit IRC16:58
corvuspabelanger, mordred: could you +3 https://review.openstack.org/527548 ?16:59
*** sree has joined #openstack-infra16:59
*** rkukura has joined #openstack-infra16:59
*** derekh_ has joined #openstack-infra17:00
*** smatzek has joined #openstack-infra17:02
*** derekh has quit IRC17:03
*** e0ne has quit IRC17:03
*** felipemonteiro__ has quit IRC17:04
*** ralonsoh_ has joined #openstack-infra17:04
*** caphrim007 has joined #openstack-infra17:04
*** hrybacki is now known as hrybacki_mtg17:04
pabelangerdone17:05
*** jamesmcarthur has joined #openstack-infra17:06
*** ralonsoh has quit IRC17:08
*** jamesmcarthur has quit IRC17:09
*** caphrim007 has quit IRC17:09
*** jamesmca_ has joined #openstack-infra17:09
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Remove nb03 / nb04 servers  https://review.openstack.org/53580417:09
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Bring online nl04.o.o  https://review.openstack.org/53645617:09
pabelangerinfra-root: if you would mind reviewing ^. Allows nl04.o.o to talk with zookeeper17:09
pabelangerand removes nb03 / nb0417:09
*** rkukura has quit IRC17:10
*** pcaruana has quit IRC17:11
*** sree has quit IRC17:11
*** jpena|brb is now known as jpena17:11
*** mlavalle has joined #openstack-infra17:17
*** janki has quit IRC17:17
mlavalleAre there plans to rename / migrate http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/zuul-legacy-project-templates.yaml#n177 to just 'tempest-dsvm-py35'?17:18
*** armax has quit IRC17:18
*** armax has joined #openstack-infra17:20
*** rlandy is now known as rlandy|brb17:20
mlavalleor 'tempest-py35'?17:21
toskymlavalle: tempest jobs are being rewritten; there is already a tempest-full which makes use of the base devstack-tempest job17:22
*** smatzek has quit IRC17:22
*** smatzek has joined #openstack-infra17:23
mlavalletosky: is the py35 job in the works?17:23
toskymlavalle: so unless I misunderstood andreaf's plans, the legacy jobs won't be migrated, but rewritten, and I don't know the status of the py35 job17:23
tosky(anyway, apart from the specific case, migrated jobs should not use dsvm in the name)17:24
*** trown|ruck is now known as trown|lunch17:24
mlavalleandreaf: any insights on the re-writing / re-naming of the tempest py35 job?17:24
mlavallethanks for the update tosky :-)17:25
toskyreference for job naming: https://docs.openstack.org/infra/manual/drivers.html#v3-naming17:26
*** ekcs has quit IRC17:26
mlavalletosky: thanks17:26
*** smatzek_ has joined #openstack-infra17:27
openstackgerritSagi Shnaidman proposed openstack/diskimage-builder master: Remove RH1 check OVB jobs from configuration  https://review.openstack.org/53648917:28
*** smatzek has quit IRC17:28
*** kjackal has quit IRC17:29
openstackgerritJames E. Blair proposed openstack-infra/puppet-mailman master: Create per-site template directories  https://review.openstack.org/53584817:29
*** dprince has quit IRC17:32
*** jpich has quit IRC17:32
*** smatzek_ has quit IRC17:33
openstackgerritMerged openstack-infra/system-config master: Add website hosting infrastructure  https://review.openstack.org/52754817:33
corvus#status log deleted all contents in /afs/openstack.org/docs/draft at request of pkovar and AJaeger17:33
openstackstatuscorvus: finished logging17:33
*** rlandy|brb is now known as rlandy17:33
AJaegermlavalle: the work is done in the tempest repo AFAIK, best ask on #openstack-qa17:34
*** smatzek has joined #openstack-infra17:35
*** hashar has joined #openstack-infra17:35
*** caphrim007 has joined #openstack-infra17:36
*** caphrim007 has quit IRC17:37
*** caphrim007 has joined #openstack-infra17:38
efriedqq y'all: if I've got a series with +Ws, and the bottom one fails (bogusly) in the gate, all the ones above it get a -2 "This change depends on a change that failed to merge."17:41
AJaegerinfra-root, can we remove tempest-lib installation from devstack? See https://review.openstack.org/#/c/536042/17:41
AJaegerreviews welcome, please17:41
efriedSo the question is: do I need to recheck all of 'em, or will the dependents automatically re-gate once the first one merges?17:42
*** dhill_ has quit IRC17:42
*** hashar is now known as hasharAway17:42
efried(In case it matters, series in question starts here: https://review.openstack.org/#/c/526540/ )17:43
*** dhill_ has joined #openstack-infra17:43
*** jbadiapa_ has joined #openstack-infra17:46
*** dprince has joined #openstack-infra17:47
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: added new endpoint create promo code  https://review.openstack.org/53651517:50
openstackgerritMerged openstack-infra/openstackid-resources master: added new endpoint create promo code  https://review.openstack.org/53651517:51
corvusefried: recheck all17:51
efriedcorvus just on the first patch?17:51
corvusefried: no you'll need to recheck all the patches with negative verified votes17:51
efriedcorvus Ohh, not like leaving a magic comment that says "recheck all".  You mean "recheck" on all of them.  Okay, thanks.17:52
*** kien-ha has quit IRC17:58
*** shardy has quit IRC17:59
*** derekh_ has quit IRC18:00
*** ralonsoh_ has quit IRC18:02
*** wolverineav has quit IRC18:03
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: webapp: add optional admin endpoint  https://review.openstack.org/53631918:03
honzaAfter a project-config/zuul job patch is merged, how long does it take for it to become active?18:04
*** wolverineav has joined #openstack-infra18:04
*** rmcall has joined #openstack-infra18:05
AJaegerhonza: it should be active immediately18:07
*** jbadiapa_ has quit IRC18:07
honzaAJaeger: thanks!18:07
*** rkukura has joined #openstack-infra18:08
*** david-lyle has quit IRC18:09
*** dtantsur is now known as dtantsur|afk18:09
openstackgerritMerged openstack-infra/system-config master: Remove nb03 / nb04 servers  https://review.openstack.org/53580418:09
openstackgerritMerged openstack-infra/system-config master: Bring online nl04.o.o  https://review.openstack.org/53645618:09
*** felipemonteiro has joined #openstack-infra18:12
*** e0ne has joined #openstack-infra18:12
*** jamesmca_ has quit IRC18:12
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fixes on attendes endpoint  https://review.openstack.org/53652118:12
openstackgerritMerged openstack-infra/openstackid-resources master: Fixes on attendes endpoint  https://review.openstack.org/53652118:13
*** felipemonteiro_ has joined #openstack-infra18:15
*** agopi_ is now known as agopi18:15
*** e0ne has quit IRC18:16
*** jpena is now known as jpena|off18:18
*** felipemonteiro has quit IRC18:18
*** armax has quit IRC18:20
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Create openstack-tox-py36 job  https://review.openstack.org/53652718:21
*** trown|lunch is now known as trown18:21
*** armax has joined #openstack-infra18:22
*** trown is now known as trown|ruck18:22
*** sree has joined #openstack-infra18:22
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: update filter param for get promo codes  https://review.openstack.org/53652818:23
openstackgerritMerged openstack-infra/openstackid-resources master: update filter param for get promo codes  https://review.openstack.org/53652818:23
*** florianf has quit IRC18:25
*** sree has quit IRC18:27
*** felipemonteiro_ has quit IRC18:29
*** pcaruana has joined #openstack-infra18:32
pabelangernl04.o.o has been started18:33
*** kien-ha has joined #openstack-infra18:33
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fixed member serializer  https://review.openstack.org/53653018:35
dmsimardmat128: did you end up talking with jamespage ? sorry I've been a bit underwater recently..18:36
openstackgerritMerged openstack-infra/openstackid-resources master: Fixed member serializer  https://review.openstack.org/53653018:36
*** e0ne has joined #openstack-infra18:36
AJaegerdmsimard: mat128's changes merged - but followup need review18:36
dmsimardAJaeger: ah, okay, I'll get to it18:36
dmsimardthanks.18:37
AJaegerdmsimard: great18:38
*** hrybacki_mtg is now known as hrybacki18:40
mrhillsmanhey great infra folks18:41
mrhillsmandocs.openstack.org showing connection refused18:41
dmsimardmrhillsman: thanks, looking18:42
dmsimardinfra-root btw ^18:42
mordredI agree. same here18:43
mordredserver is up - apache is not18:43
mordreddmsimard: ^^18:43
pabelangeris it possible zuul-ci.org changes affected it?18:44
mordred[Mon Jan 22 17:57:27.215068 2018] [mpm_worker:notice] [pid 1590:tid 139989460993920] AH00295: caught SIGTERM, shutting down18:44
mordredis the last entry in the error.log18:44
corvuspabelanger: good idea18:44
*** ykarel|afk has quit IRC18:44
dmsimardJan 22 18:44:34 files02 apache2[5822]: SSLCertificateChainFile: file '/etc/ssl/certs/zuul-ci.org_intermediate.pem' does not exist or is empty18:44
corvusi'll modify the configuration manually18:45
corvuscan someone add to emergency file?18:45
dmsimard /etc/ssl/certs/zuul-ci.org.pem exists but not the intermediate.18:45
dmsimardcorvus: I'll add it to emergency18:46
corvusrestarted apache18:46
corvusdmsimard: thk18:46
dmsimardcorvus: files02.o.o correct ?18:46
*** esberglu has quit IRC18:46
*** florianf has joined #openstack-infra18:46
corvusdmsimard: yep18:46
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: WIP: Convert from legacy to native devstack job  https://review.openstack.org/53589918:46
dmsimarddone18:46
mordredspeaking of zuul-related ssl chain files ... https://review.openstack.org/#/c/536148/18:47
corvusoh that's a different thing18:48
mordredit is18:48
*** eharney has joined #openstack-infra18:48
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: WIP: Convert from legacy to native devstack job  https://review.openstack.org/53589918:49
mordredjust the chain file above made me remember to mentoin it ... although it doens't acutlaly seem to be needed, as https://zuul.openstack.org seems happy now18:49
*** mrtenio has quit IRC18:49
*** esberglu has joined #openstack-infra18:49
*** florianf has quit IRC18:49
*** pblaho has quit IRC18:50
*** ekcs has joined #openstack-infra18:51
*** florianf has joined #openstack-infra18:51
*** pblaho has joined #openstack-infra18:52
corvusi don't understand why that file isn't there18:52
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: WIP: Convert from legacy to native devstack job  https://review.openstack.org/53589918:55
*** links has quit IRC18:55
dmsimard#status log (dmsimard) files02.o.o was put in the emergency file pending a fix due to a missing zuul-ci.org_intermediate.pem file preventing apache from restarting properly18:56
openstackstatusdmsimard: finished logging18:56
openstackgerritDarragh Bailey proposed openstack/python-jenkins master: Migration to using requests  https://review.openstack.org/20563718:57
dmsimardmrhillsman: btw docs.o.o is back online, thanks for letting us know18:57
AJaegerdmsimard: could you review https://review.openstack.org/532543 and https://review.openstack.org/533111 for new repos as well? And also https://review.openstack.org/#/c/536042/ , please?18:57
dmsimardAJaeger: looking18:57
*** rkukura has quit IRC18:57
*** myoung|biab is now known as myoung18:57
corvusoh, i see the problem.18:57
corvusfix in a minute.18:58
AJaegerthanks, dmsimard18:58
dmsimardAJaeger: don't we have something that checks if pypi is registered and we have access to it ? is that jeepyb ? in relation to https://review.openstack.org/#/c/532543/18:59
*** oidgar has joined #openstack-infra18:59
AJaegerdmsimard: no, we don't have that.19:00
mordreddmsimard: I wrote a jeepyb patch to add that19:00
openstackgerritMerged openstack-infra/project-config master: Add charm-neutron-api-genericswitch project to Zuul  https://review.openstack.org/53160719:00
AJaegerdmsimard: mordred was working on https://review.openstack.org/#/c/531898/ but that needs some more work to pass.19:00
dmsimardmordred: ah I knew that rung a bell19:00
mordreddmsimard: https://review.openstack.org/#/c/531898/19:00
mordredyah19:00
AJaegermordred: it does not pass the tests ;(19:00
mordredyah. need to fix that19:01
*** kien-ha has quit IRC19:01
*** jamesmca_ has joined #openstack-infra19:01
*** jkilpatr has quit IRC19:02
*** pkovar has quit IRC19:02
*** rlandy is now known as rlandy|biab19:03
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Fix zuul-ci.org website def  https://review.openstack.org/53653819:03
corvusmordred, dmsimard, pabelanger: ^ that should correct the current issue and make safe from the same happening in future19:04
mrhillsmandmsimard welcome, thanks infra-root for quick response19:04
*** hasharAway has quit IRC19:04
*** lucasagomes is now known as lucas-afk19:04
*** david-lyle has joined #openstack-infra19:07
openstackgerritMerged openstack-infra/project-config master: Add charm-ironic to zuul projects  https://review.openstack.org/53160919:08
*** amoralej is now known as amoralej|off19:09
*** Goneri has quit IRC19:09
*** electrofelix has quit IRC19:10
*** dayou has quit IRC19:12
*** dayou has joined #openstack-infra19:13
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Remove OVH provider from nl02.o.o  https://review.openstack.org/53653919:13
pabelangerconfig-core: ^ cleans up nl02.o.o now that nl04.o.o is online19:13
*** sree has joined #openstack-infra19:14
*** wolverineav has quit IRC19:14
*** hashar has joined #openstack-infra19:15
*** jamesmca_ has quit IRC19:15
pabelangerconfig-core: sorry, https://review.openstack.org/536036/ seems 536539 is a dupe19:16
*** jkilpatr has joined #openstack-infra19:16
*** jamesmcarthur has joined #openstack-infra19:16
*** jamesmcarthur has quit IRC19:17
*** jamesmcarthur has joined #openstack-infra19:17
*** rkukura has joined #openstack-infra19:18
openstackgerritMerged openstack-infra/project-config master: Adding jobs to new tempest-stress  https://review.openstack.org/53311119:18
*** sree has quit IRC19:19
*** armaan has quit IRC19:19
*** rosmaita has quit IRC19:19
*** armaan has joined #openstack-infra19:20
pabelangerI'm going to start work on replacing nl02.o.o with 2GB instances, last step needed in splitting launchers up19:23
*** sree has joined #openstack-infra19:25
*** rosmaita has joined #openstack-infra19:25
*** hashar has quit IRC19:25
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Fix race in test_provider_removal  https://review.openstack.org/53654019:26
smcginnisInfra folks (cc: dhellmann) - Are there any planned upcoming changes that may impact releases that we should be aware of?19:26
*** camunoz has joined #openstack-infra19:27
openstackgerritMerged openstack-infra/project-config master: Add jobs for n-g-s tempest-plugin  https://review.openstack.org/53254319:27
* AJaeger will be back later19:27
*** pcichy has quit IRC19:27
*** hashar has joined #openstack-infra19:27
pabelangersmcginnis: I don't believe we have anything scheduled, we recently rebuilt zuul01.o.o to have more memory to help with existing outages we have been having19:28
smcginnispabelanger: OK, great. Thanks!19:29
tonybprometheanfire: all projects that use constratints break when we EOL a branch until we finish the static publication of constratints work, which is next cycle now (at the soonest)19:29
mlavalleAJaeger: thanks!19:29
*** sree has quit IRC19:30
tonybAJaeger: I do need to finish the newton-eol.  I'll try to validate that it's just nova and ironic today and if so request an infra core do it or add me to boostratppers so I can do it19:31
*** wolverineav has joined #openstack-infra19:31
*** hashar has quit IRC19:32
*** rkukura has quit IRC19:32
*** rkukura has joined #openstack-infra19:34
openstackgerritJames E. Blair proposed openstack-infra/puppet-mailman master: Create per-site template directories  https://review.openstack.org/53584819:34
*** rlandy|biab is now known as rlandy19:37
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: import blog post "Things that StoryBoard does differently"  https://review.openstack.org/53441219:38
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: separate worklist instructions from theory  https://review.openstack.org/53441319:38
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: fix document cross-link syntax  https://review.openstack.org/53441419:38
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: fix syntax for toctree  https://review.openstack.org/53441519:38
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: fix literal block markup for blocks within bullet lists  https://review.openstack.org/53441619:38
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: escape an example URL as literal text to avoid build warning  https://review.openstack.org/53441719:38
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: fix field list markup syntax  https://review.openstack.org/53441819:38
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: highlight notes about permissions on API endpoints  https://review.openstack.org/53441919:38
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: suppress warnings from wsmeext.sphinxext  https://review.openstack.org/53487219:38
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: treat doc build warnings as errors  https://review.openstack.org/53487319:38
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: import blog post "Mapping Launchpad to StoryBoard"  https://review.openstack.org/53442019:38
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: import blog post "Why StoryBoard for OpenStack?"  https://review.openstack.org/53442119:38
*** rkukura has quit IRC19:40
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: Clean held nodes automatically after configurable timeout  https://review.openstack.org/53629519:40
*** rkukura has joined #openstack-infra19:40
*** kien-ha has joined #openstack-infra19:40
*** pcaruana has quit IRC19:43
*** dhill_ has quit IRC19:44
*** sree has joined #openstack-infra19:45
*** dhill_ has joined #openstack-infra19:46
*** rkukura has quit IRC19:47
*** sshnaidm is now known as sshnaidm|off19:48
openstackgerritMerged openstack-infra/system-config master: Add ssl chain file to zuul  https://review.openstack.org/53614819:50
*** sree has quit IRC19:50
*** agopi has quit IRC19:51
AJaegertristanC: does https://review.openstack.org/#/c/534431/ look ok now?19:54
*** slaweq has joined #openstack-infra19:57
*** agopi has joined #openstack-infra19:57
AJaegerinfra-root, could you review frickler's devstack-gate change https://review.openstack.org/#/c/529000/ , please?19:57
pabelangernew nl02.o.o (2GB) online19:58
pabelangerwaiting for firewalls to be updated with new DNS19:58
AJaegerpabelanger: do we need grafana updates for nl03 and nl04?19:59
*** dave-mcc_ is now known as dave-mcowan19:59
*** dave-mcowan is now known as dave-mccowan19:59
*** dave-mccowan is now known as dave-mcc_20:00
*** rkukura has joined #openstack-infra20:00
pabelangerAJaeger: yah, firewall rules for them should be in place20:00
pabelangerbut don't think we need any changes to grafana itself20:00
*** rkukura has quit IRC20:01
*** armax has quit IRC20:02
AJaegerpabelanger: looking at http://grafana.openstack.org/dashboard/db/nodepool , I don't see any nl01 etc at all. should we add them somehow?20:03
*** armax has joined #openstack-infra20:03
pabelangerAJaeger: I'm not sure if we track that data in statsd per launcher, would need to look20:04
openstackgerritMerged openstack-infra/project-config master: Remove OVH from nl02.o.o  https://review.openstack.org/53603620:04
*** dsariel has quit IRC20:04
*** dsariel_ has quit IRC20:04
*** rfolco|rover is now known as rfolco|off20:04
*** harlowja has joined #openstack-infra20:04
AJaegercorvus, I think we can remove the name from the project stanza for in-tree jobs in the manual - see https://review.openstack.org/536122 . What do you think?20:07
*** armax has quit IRC20:08
*** sree has joined #openstack-infra20:12
corvusAJaeger: ++20:12
AJaegerthanks20:13
*** ldnunes has quit IRC20:15
*** sree has quit IRC20:17
*** sshnaidm|off has joined #openstack-infra20:19
*** Goneri has joined #openstack-infra20:20
*** jkilpatr has quit IRC20:25
pabelangernodepool-launcher started on nl02.o.o20:32
*** agopi has quit IRC20:33
*** peterlisak has quit IRC20:33
*** johnthetubaguy has quit IRC20:33
*** peterlisak has joined #openstack-infra20:34
*** agopi has joined #openstack-infra20:34
*** oidgar has quit IRC20:36
*** witek has quit IRC20:36
*** swest has quit IRC20:38
*** smatzek has quit IRC20:38
*** swest has joined #openstack-infra20:39
*** witek has joined #openstack-infra20:39
*** johnthetubaguy has joined #openstack-infra20:40
*** ijw has quit IRC20:41
*** ijw has joined #openstack-infra20:42
*** ijw_ has joined #openstack-infra20:43
*** sree has joined #openstack-infra20:44
*** florianf has quit IRC20:45
*** r-daneel has quit IRC20:46
*** ijw_ has quit IRC20:47
*** sree has quit IRC20:49
smcginnisWe had a release job fail with a 503 from pypi. Manual checks now are fine.20:52
pabelangerwhich provider?20:52
smcginnisCould we get this one reenqueued? http://logs.openstack.org/17/17fe24c0449ef2067ed7c7e0e51397711becef0d/release/release-openstack-python/1b3f482/job-output.txt.gz#_2018-01-22_20_41_09_05095220:52
pabelangeroh, pypi directly20:52
smcginnispabelanger: Yep.20:52
smcginnispabelanger: I thought it was a network failure at first, but that looks like it came right from pypi.org.20:52
pabelangersure, give me a few moments. need to remember how to do it20:53
smcginnispabelanger: Thanks!20:53
*** jkilpatr has joined #openstack-infra20:53
*** andreas_s has joined #openstack-infra20:56
pabelangerzuul enqueue-ref --tenant openstack --trigger gerrit --pipeline release --project openstack/networking-ovn --ref 17fe24c0449ef2067ed7c7e0e51397711becef0d20:59
pabelangersmcginnis: ^I think that is our command21:00
*** andreas_s has quit IRC21:00
pabelangerokay, change added to pipeline21:01
smcginnispabelanger: Nice, I will make a note of that in case it ever comes up again.21:01
*** r-daneel has joined #openstack-infra21:04
*** sree has joined #openstack-infra21:04
*** dave-mcc_ has quit IRC21:05
*** jamesmcarthur has quit IRC21:06
*** dbecker has quit IRC21:07
*** dbecker has joined #openstack-infra21:08
*** sree has quit IRC21:09
*** dbecker has quit IRC21:09
*** jamesmcarthur has joined #openstack-infra21:14
*** esberglu has quit IRC21:16
*** esberglu has joined #openstack-infra21:16
*** esberglu_ has joined #openstack-infra21:19
*** kien-ha has quit IRC21:20
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: WIP: Convert from legacy to native devstack job  https://review.openstack.org/53589921:20
*** esberglu has quit IRC21:20
*** e0ne has quit IRC21:22
*** jcoufal has quit IRC21:23
smcginnispabelanger: Well, last job succeeded, but next one failed now.21:24
smcginnisI've never seen this one: http://logs.openstack.org/release/git.openstack.org/openstack/networking-ovn/17fe24c0449ef2067ed7c7e0e51397711becef0d/announce-release/3e3f451/job-output.txt.gz#_2018-01-22_21_22_02_54795921:25
smcginnisLooks like an ansible job formatting issue, but we've used this many times.21:25
smcginnisPlaybook was last updated Aug 29, 2017.21:27
smcginnisSame issue with the propose-update-constraint job: http://logs.openstack.org/release/git.openstack.org/openstack/networking-ovn/17fe24c0449ef2067ed7c7e0e51397711becef0d/propose-update-constraints/9a44702/job-output.txt.gz#_2018-01-22_21_21_59_24759321:28
openstackgerritMerged openstack-infra/nodepool master: license: remove dos line break  https://review.openstack.org/53553121:28
pabelangersmcginnis: looking21:29
fungi#status log restarted openstack-paste (lodgeit) service on paste.openstack.org as it was timing out responding to proxied requests from apache21:29
openstackstatusfungi: finished logging21:29
*** kgiusti has left #openstack-infra21:29
fungismcginnis: pabelanger: i'm not really here (just checking e-mail from the hotel room real quick) but here's a dirty script i've been using the last week or so to generate the requisite enqueue-ref command line from a previous job's base url: http://paste.openstack.org/show/65051121:29
pabelangersmcginnis: oh, it could be of the way I enqueued the change21:30
pabelangersmcginnis: I didn't add tag information21:30
*** dave-mccowan has joined #openstack-infra21:30
fungiwe could presumably generalize that script to also support enqueue commands for change-based refs and then include it in the zuul repo or something21:30
pabelangerlet me see how that affected the other job21:30
pabelangerfungi: yah21:31
pabelangerit is looking for tag: ref: refs/tags/3.0.121:31
fungipabelanger: the script in my paste (above) should get all the parameters correctly included21:31
pabelangerin inventory file, which wasn't part of equeue-ref command21:31
smcginnispabelanger, fungi: Thanks!21:31
fungiwith proper formatting21:31
*** armax has joined #openstack-infra21:32
pabelangerfungi: good script, will use that next time21:33
pabelangersmcginnis: is the version you expect on pypi?21:34
* smcginnis looks...21:34
pabelangersmcginnis: I'm not sure we can enqueue the jobs again for release announcement21:34
pabelangersince it was uploaded21:35
smcginnisYeah, not too concerned about the announcement, but the requirements proposal would be nice.21:35
smcginnisOf course that could be done manually as well.21:35
pabelangeryah, lets confirm it is the right version, incase tag info was needed21:35
pabelangerif not correct, I'll use fungi script this time21:36
fungiyep, once it's on pypi you can't reenqueue that tag ref because the pypi upload will fail from that point on (pypi disallows replacing uploaded files)21:36
pabelangeryah21:36
SamYapleare there known zuul issues at this time? (job hasnt arrived in check queue yet)21:36
*** jamesmcarthur has quit IRC21:36
*** jamesmcarthur has joined #openstack-infra21:36
*** jamesmcarthur has quit IRC21:37
pabelangerSamYaple: don't believe so, but queue length is growing21:37
pabelangergive it another minute or so21:37
smcginnispabelanger: Hmm, nope, not on pypi. 3.0.0 is there, but this was for 3.0.1.21:37
SamYaplesure. ill check back (change is 5m old right now)21:37
smcginnisLooks like we ended up with a 4.0.0.0b3.dev7821:38
fungismcginnis: pabelanger: should be safe to reenqueue in that case21:38
pabelangeryah, I see that now21:38
pabelangerokay21:38
* fungi disappears back into the aether21:38
smcginnisfungi: Thanks21:40
SamYaplepabelanger: it might be zuul.o.o, now thats breaking for me21:41
SamYapleah no its all good now21:41
SamYapleshould have just waited a few more seconds21:42
pabelangeryah, queue is empty now21:42
dmsimardcorvus: btw for your ssl patch http://logs.openstack.org/38/536538/1/check/legacy-infra-puppet-apply-3/f2b6e94/openstack-infra/system-config/applytest/puppetapplytest30.final.out.FAILED21:42
pabelangersmcginnis: okay, enqueued again using fungi script21:42
pabelangerI am not sure what to do about 4.0.0.0b3.dev78 on pypi right now21:43
smcginnisFungiScript. Like JavaScript but better.21:43
smcginnis:)21:43
SamYapleheh21:43
smcginnispabelanger: We can probably just leave it. We will get a final Queens version soon.21:44
smcginnispabelanger: Unless you know how to delete that from there.21:44
*** wolverineav has quit IRC21:45
pabelangerdmsimard: corvus: I see the issue now, left comments21:45
pabelangersmcginnis: yah, not sure delete is best, since we usually complain that others do that and it breaks things :)21:46
smcginnispabelanger: IIUC, it's just a duplicate of their last milestone release, so I would assume it's probably safe to leave it.21:46
pabelangerack21:47
openstackgerritMarcin Juszkiewicz proposed openstack/diskimage-builder master: GPT partitioning support  https://review.openstack.org/53349021:50
openstackgerritMarcin Juszkiewicz proposed openstack/diskimage-builder master: Don't fstrim vfat partitions  https://review.openstack.org/53496921:50
openstackgerritMarcin Juszkiewicz proposed openstack/diskimage-builder master: move to 'sgdisk' for GPT  https://review.openstack.org/53659921:50
openstackgerritMarcin Juszkiewicz proposed openstack/diskimage-builder master: bootloader: handle grub-efi better  https://review.openstack.org/53660021:50
openstackgerritMarcin Juszkiewicz proposed openstack/diskimage-builder master: change default image into GPT with ESP and BSP  https://review.openstack.org/53660121:50
*** edmondsw has quit IRC21:52
*** dave-mcc_ has joined #openstack-infra21:52
*** dave-mccowan has quit IRC21:52
dmsimardIs this a javascript gerrit CI thing issue ? the no-op that last ran didn't overwrite the previous job results https://review.openstack.org/#/c/535339/21:53
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Fix zuul-ci.org website def  https://review.openstack.org/53653821:55
corvuspabelanger, mordred: ^ fixed21:55
openstackgerritMerged openstack-infra/project-config master: Add api-ref jobs for Workflow service (mistral)  https://review.openstack.org/53629421:58
*** rosmaita_ has joined #openstack-infra22:00
*** yamahata has joined #openstack-infra22:00
*** trown|ruck is now known as trown|outtypewww22:02
*** rosmaita has quit IRC22:02
*** agopi_ has joined #openstack-infra22:03
*** jtomasek has quit IRC22:03
*** agopi has quit IRC22:05
openstackgerritMerged openstack-infra/project-config master: Remove jobs from tempest-lib  https://review.openstack.org/52952422:10
openstackgerritMerged openstack-infra/project-config master: add required projects for vmware-nsx periodic with neutron-lib  https://review.openstack.org/53573222:10
openstackgerritMerged openstack-infra/project-config master: add charm-panko to project-config  https://review.openstack.org/53327322:10
*** aviau has quit IRC22:11
*** yamahata has quit IRC22:11
*** myoung is now known as myoung|bbl22:12
*** myoung|bbl is now known as myoung22:12
*** aviau has joined #openstack-infra22:13
*** jaosorior has quit IRC22:13
*** slaweq has quit IRC22:17
*** openstackgerrit has quit IRC22:18
*** rcernin has joined #openstack-infra22:19
*** sree has joined #openstack-infra22:20
*** jamesmcarthur has joined #openstack-infra22:21
*** openstackgerrit has joined #openstack-infra22:21
openstackgerritMerged openstack-infra/devstack-gate master: Remove tempest-lib  https://review.openstack.org/53604222:21
dmsimardOh, I just got one of those emails, am I santa ? https://xkcd.com/838/22:23
*** sree has quit IRC22:25
cloudnullpabelanger: clarkb: mordred: ping -- qq does infra have any historical data from the OSIC? like node count, build times, etc?22:26
cloudnulllike old grafana / graphite logs stashed somewhere?22:27
mordredcloudnull: http://graphite.openstack.org/22:27
*** claudiub has quit IRC22:27
mordredcloudnull: stats.nodepool.launch.provider.osic-* ... also stats.timers.launch.provider.osic-* and and stats.timers.task.osic-*22:29
mordredcloudnull: are all good places to go look22:29
cloudnullah ha!22:29
cloudnullmost excellent.22:29
cloudnullthanks mordred22:29
mordredcloudnull: also - nice demolition work in your house :)22:29
cloudnullthanks!22:30
cloudnullthat was a bunch of fun22:30
*** jamesmcarthur has quit IRC22:30
cloudnullwe22:30
cloudnull**we're soon to have an even more open floor plan :)22:30
mordredcloudnull: woot!22:31
*** jamesmcarthur has joined #openstack-infra22:32
*** ijw has joined #openstack-infra22:33
openstackgerritDavid Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Resolve Ansible variable precedence issue with include_vars  https://review.openstack.org/53661522:33
dmsimardmordred: ^ I think this is what we want22:33
mordreddmsimard: ++ wow. what a long-lived task ... I've forgotten what we were even trying to do originally now :)22:35
dmsimardmordred: make emit-job-header awesome22:36
mordredoh right!22:36
dmsimardmordred: it's preventing us from landing https://review.openstack.org/#/c/514489/ and https://review.openstack.org/#/c/532304/22:36
mordredadding a debug line :)22:36
mordreddmsimard: thanks for sticking with that22:36
dmsimardmordred: I'll rebase my patch and maybe add ansible_version to it ?22:36
dmsimardmordred: np, it's been in the back burner for way too long.. trying to stick my head above the water :)22:37
openstackgerritDavid Moreau Simard proposed openstack-infra/zuul-jobs master: Revert "Revert "Add zuul.{pipeline,nodepool.provider,executor.hostname} to job header""  https://review.openstack.org/51448922:38
*** smatzek has joined #openstack-infra22:38
*** jaosorior has joined #openstack-infra22:39
openstackgerritDavid Moreau Simard proposed openstack-infra/zuul-jobs master: Add Ansible version to job header  https://review.openstack.org/53230422:42
*** smatzek has quit IRC22:43
openstackgerritDavid Moreau Simard proposed openstack-infra/zuul-jobs master: Revert "Revert "Add zuul.{pipeline,nodepool.provider,executor.hostname} to job header""  https://review.openstack.org/51448922:43
openstackgerritDavid Moreau Simard proposed openstack-infra/zuul-jobs master: Add Ansible version to job header  https://review.openstack.org/53230422:43
dmsimardmordred: ^ rebased your patch on top of mine22:44
mordredossum22:44
dmsimardhttps://review.openstack.org/#/c/529428/ has 5 +2s, pabelanger do you want to do the final +3 ?22:46
dmsimardor ianw ^22:47
*** agopi__ has joined #openstack-infra22:50
*** wolverineav has joined #openstack-infra22:53
*** agopi_ has quit IRC22:53
*** wolverineav has quit IRC22:59
ianwdespite my attempts at adding append etc, i don't think i've ever managed to keep bash from randomly destroying history across different sessions23:01
pabelangerdmsimard: if you want +3 and monitor, that fine by me23:04
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Fix a copy-pasta in a comment  https://review.openstack.org/53662023:05
*** bobh has quit IRC23:07
*** bobh has joined #openstack-infra23:08
*** eharney has quit IRC23:09
*** bobh has quit IRC23:12
*** dprince has quit IRC23:15
*** mlavalle has quit IRC23:23
dmsimardinfra-root: the periodic zuul scheduler backups from https://review.openstack.org/#/c/532955/ aren't working because it tries to fetch zuul01.o.o over https and the certificate doesn't match -- I'll send a patch to fix after dinner23:23
*** mrhillsman has quit IRC23:26
*** jungleboyj has quit IRC23:26
*** tpsilva has quit IRC23:26
*** jungleboyj has joined #openstack-infra23:26
*** tpsilva has joined #openstack-infra23:26
*** betherly has quit IRC23:26
*** kmalloc has quit IRC23:26
*** DuncanT has quit IRC23:26
*** niedbalski has quit IRC23:26
*** gtmanfred has quit IRC23:26
*** icey has quit IRC23:26
*** mrhillsman has joined #openstack-infra23:26
*** rajinir has quit IRC23:26
*** kmARC has quit IRC23:26
*** betherly has joined #openstack-infra23:27
*** icey has joined #openstack-infra23:27
*** kmalloc has joined #openstack-infra23:27
*** kmARC has joined #openstack-infra23:27
*** rajinir has joined #openstack-infra23:27
*** gtmanfred has joined #openstack-infra23:27
*** mancdaz has quit IRC23:27
*** tdasilva has quit IRC23:27
*** lennyb has quit IRC23:27
*** DuncanT has joined #openstack-infra23:27
openstackgerritDavid Moreau Simard proposed openstack-infra/zuul master: Add support for dumping queues from a status.json file  https://review.openstack.org/53662223:28
*** tdasilva has joined #openstack-infra23:28
*** dhill_ has quit IRC23:28
*** Adri2000 has quit IRC23:28
*** thingee has quit IRC23:28
*** dhellmann has quit IRC23:28
*** lennyb has joined #openstack-infra23:28
*** Shrews has quit IRC23:28
*** dhellmann has joined #openstack-infra23:29
openstackgerritDavid Moreau Simard proposed openstack-infra/system-config master: Adjust documentation for the new --file zuul-changes.py parameter  https://review.openstack.org/53662323:29
*** niedbalski has joined #openstack-infra23:29
*** mancdaz has joined #openstack-infra23:29
*** Adri2000 has joined #openstack-infra23:29
*** Adri2000 has quit IRC23:29
*** Adri2000 has joined #openstack-infra23:29
*** bandini has quit IRC23:29
*** bandini has joined #openstack-infra23:32
*** Shrews has joined #openstack-infra23:35
cloudnullthanks again mordred, the info in stats.nodepool was super helpful.23:38
*** ganso has quit IRC23:38
mordredcloudnull: woot! glad it was helpful23:41
ianwmordred: as i sit here waiting for another image upload ... i wonder if it is worth restoring https://review.openstack.org/#/c/474425/ which wraps image upload for a progress option?  i don't want to invest too much time if it's never going to fly23:44
*** efried is now known as efried_back_wed23:46
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Add change information to Build Completed log message  https://review.openstack.org/53662923:49
*** bobh has joined #openstack-infra23:55
*** thingee has joined #openstack-infra23:55
*** openstack has quit IRC23:55
*** openstack has joined #openstack-infra23:56
*** ChanServ sets mode: +o openstack23:56
*** jamesmcarthur has quit IRC23:57
*** agopi__ has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!