| sean-k-mooney | there is no way to do that in zuul either unfortunetly | 00:00 |
|---|---|---|
| clarkb | sean-k-mooney: one issue with trying to do it in zuul is knowing how to approximate the billing of $cloud | 00:01 |
| clarkb | its actually really easy to get wrong cloud to cloud | 00:01 |
| clarkb | from zuul's end | 00:01 |
| sean-k-mooney | clarkb: well seeting a quota on hours of vm time per node poole provider per month woudl be a good start | 00:01 |
| sean-k-mooney | but you also dont want ot have to keep track of that in nodepool or zuul really | 00:02 |
| *** jamesmcarthur has quit IRC | 00:02 | |
| sean-k-mooney | if i had a better place to put my rackmount server i would have plenty of capastiy at home for what i want to test. as it is i still should have enough form manually triggred ci via gerrit comments | 00:04 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: vexxhost: tweak nodepool settings https://review.openstack.org/623605 | 00:04 |
| mnaser | clarkb: ^ looks like we were *mostly* there anyways :) | 00:04 |
| sean-k-mooney | + maybe some minimal auto triigered jobs. | 00:05 |
| sean-k-mooney | so out of interest is there a way for me to request a job to run on vexhost since ye support nested virt | 00:06 |
| *** calbers has quit IRC | 00:12 | |
| *** openstack has quit IRC | 00:12 | |
| *** openstack has joined #openstack-infra | 00:19 | |
| *** ChanServ sets mode: +o openstack | 00:19 | |
| mnaser | corvus: yeah seems like this might be easier to be defined in documentation than code | 00:19 |
| corvus | mnaser: the question is, does the pipeline vote, and if so, as what user? | 00:19 |
| *** sshnaidm|off has joined #openstack-infra | 00:19 | |
| mnaser | corvus: well, i'm thinking have a restriction in a trusted repo that controls which nodesets can enter a pipeline | 00:19 |
| corvus | mnaser: if you want the pipeline to vote, it can't vote as zuul, because the zuul user only gets one vote, so you need a second user in zuul, and then it's really confusing to try to keep that straight in zuul's configuration. | 00:20 |
| mnaser | yeah but i see how a +1 would never land if that cloud is not there | 00:20 |
| corvus | mnaser: like i said, i *think* the technical capability to do that is there, but i don't believe it's gotten real-world testing yet. | 00:20 |
| *** openstackgerrit has quit IRC | 00:20 | |
| *** mdrabe has quit IRC | 00:20 | |
| mnaser | okay im seeing why this can be hard | 00:20 |
| mnaser | lol | 00:20 |
| corvus | mnaser: it's easier if the pipeline doesn't vote -- that's just like the experimental pipeline. | 00:21 |
| sean-k-mooney | mnaser: that would stop poject define there own nodes for i ned 2 nodes vs 1 vs 5 for this job which they can do today i belive | 00:21 |
| corvus | but apparently, no one looks at comments, only votes. :| | 00:21 |
| sean-k-mooney | corvus: ya | 00:21 |
| mnaser | corvus: unfortunately that's the case, and in my experience nv jobs are largely ignored :( | 00:21 |
| sean-k-mooney | well i do | 00:21 |
| sean-k-mooney | mnaser: not always. i check them on os-vif and i check some of them on nova | 00:22 |
| mnaser | sean-k-mooney: right, but for the most part i feel like they're ignored but that might just be me | 00:22 |
| sean-k-mooney | there are cis i dont trust and when they fail i decide if i want to check them or not | 00:22 |
| mnaser | oh and also does any infra-root know anything about inap getting centos 7.6 images up | 00:23 |
| mnaser | we're still seeing failures because of the mismatches | 00:23 |
| corvus | as far as nested virt in particular goes -- i'm pretty sure if we had any capability for that at all, all the projects would rely on it post-haste. | 00:23 |
| sean-k-mooney | corvus: well the fact is we test nova in a way that will never be depoyled in production | 00:24 |
| sean-k-mooney | e.g. no one will un x86 guest on x86 host with qemu instead of kvm | 00:24 |
| corvus | sean-k-mooney: sure, i don't think the status quo is good, it's just possible. :) | 00:24 |
| sean-k-mooney | that is true too | 00:24 |
| sean-k-mooney | one thing i have been think about is makeing qemu mode faster | 00:25 |
| clarkb | mnaser: yes, see scrollback re centos 7.6. Inap is failing to upload images due to what I assume is a bug in openstacksdk | 00:25 |
| sean-k-mooney | qemu has support for multi threaded tcg backend but nova does not | 00:25 |
| clarkb | mnaser: http://paste.openstack.org/show/736850/ | 00:26 |
| clarkb | mnaser: all other clouds should have 7.6 now | 00:26 |
| mnaser | clarkb: darn, i'd hate for us to disable another cloud but that's just gonna keep making us fail randomly unfortunately | 00:26 |
| clarkb | mnaser: I was hoping that mordred would be around and recognize the issue | 00:26 |
| clarkb | its a weird one since its mad the state is changing to the same state | 00:26 |
| corvus | clarkb: did we re-upgrade openstacksdk? | 00:26 |
| clarkb | corvus: the buildersa re running 0.19.0 | 00:27 |
| mnaser | yeah its a tad weird | 00:27 |
| clarkb | or at least the one I checked is /me checks the other | 00:27 |
| clarkb | ya both are 0.19.0 | 00:27 |
| corvus | well, we have 3 don't we? (but 3 hasn't been restarted in a long time) | 00:30 |
| corvus | and it's not uploading that image anyway | 00:30 |
| *** bobh has joined #openstack-infra | 00:32 | |
| *** ssbarnea|rover has quit IRC | 00:33 | |
| clarkb | 3 is arm only and no centos there | 00:35 |
| *** bobh has quit IRC | 00:37 | |
| clarkb | we saw a similar thing with glance in rax where we were passing all the properties through each time and then it would be mad you were trying to update those properties | 00:37 |
| clarkb | but in the regular v2 upload path we are using with inap we don't seem to pass any of that info, just the actual image data | 00:38 |
| clarkb | sean-k-mooney: re how we deploy noav, I think a key thing to remember is we aren't testing libvirt and kvm and qemu, we are testing nova's interaction with libvirt | 00:39 |
| clarkb | sean-k-mooney: and whether you use qemu or kvm the overlap in that interaction is significant | 00:39 |
| clarkb | so while not ideal, its definitely still useful to test | 00:39 |
| clarkb | corvus: mnaser we could try downgrading to 0.17.2 ? | 00:40 |
| clarkb | hrm could it also be keystoneauth1 | 00:41 |
| mnaser | 240 commits between those two.. i dunno if a fix for it has helped | 00:41 |
| sean-k-mooney | clarkb: there is a resonable overlay but we also diverge alot in some areas | 00:41 |
| corvus | oh right, we could still be using the new keystoneauth1? | 00:41 |
| clarkb | corvus: the most recent keystoneauth1 from the 6th is installed | 00:41 |
| clarkb | but the builder was started on the 5th | 00:42 |
| corvus | clarkb: that is supposed to fix the rax problems. it's probably worth just restarting with that before we try downgrading | 00:42 |
| sean-k-mooney | well yes and no it not that we diverge it that several feautre require kvm to be ebaled so they cant be tested in the gate currently but that is fine. | 00:42 |
| clarkb | ok I'll restart the service on nb01 and nb02 now | 00:42 |
| clarkb | sean-k-mooney: ya the downside is we can't test the specific features in one but not the other | 00:42 |
| clarkb | that nova exposes | 00:43 |
| sean-k-mooney | yep which rules all the nfv features | 00:43 |
| sean-k-mooney | by ya i know we can only do so much with the hardware we have | 00:43 |
| sean-k-mooney | the upstream ci is still the best ci env i have ever worked with | 00:44 |
| clarkb | nb02 is restarted but nb01 is waiting for a build to clean up | 00:45 |
| clarkb | ok and now both are restarted | 00:46 |
| *** diablo_rojo has quit IRC | 00:47 | |
| *** eumel8 has quit IRC | 00:49 | |
| clarkb | mnaser: corvus: if that keystoneauth1 update has helped fix things then it should be happy in the near future | 00:52 |
| clarkb | if not then we'll need to figure out what is going on with glance and sdk | 00:52 |
| *** bobh has joined #openstack-infra | 01:09 | |
| *** d0ugal has quit IRC | 01:12 | |
| *** bobh has quit IRC | 01:13 | |
| *** d0ugal has joined #openstack-infra | 01:25 | |
| *** bobh has joined #openstack-infra | 01:28 | |
| clarkb | ok we still have the same error uploading to inap so keystoneauth1 update wasn't the fix | 01:29 |
| clarkb | mordred: ^ if you end up seeing this help would be appreciated. I need to go sort out dinner now though | 01:30 |
| *** bobh has quit IRC | 01:33 | |
| *** rh-jelabarre has quit IRC | 01:34 | |
| *** rh-jelabarre has joined #openstack-infra | 01:37 | |
| *** Swami has quit IRC | 01:38 | |
| *** jamesmcarthur has joined #openstack-infra | 01:45 | |
| *** bobh has joined #openstack-infra | 01:46 | |
| *** eernst has joined #openstack-infra | 01:48 | |
| *** bobh has quit IRC | 01:50 | |
| *** gyee has quit IRC | 01:54 | |
| *** eernst has quit IRC | 01:54 | |
| *** eernst has joined #openstack-infra | 01:54 | |
| *** ahosam has quit IRC | 01:56 | |
| *** wolverin_ has quit IRC | 02:02 | |
| *** bobh has joined #openstack-infra | 02:04 | |
| *** bobh has quit IRC | 02:08 | |
| *** jamesmcarthur has quit IRC | 02:12 | |
| *** eernst has quit IRC | 02:13 | |
| *** mrsoul has quit IRC | 02:14 | |
| *** bobh has joined #openstack-infra | 02:19 | |
| *** bobh has quit IRC | 02:24 | |
| *** anteaya has quit IRC | 02:30 | |
| *** bobh has joined #openstack-infra | 02:35 | |
| *** bobh has quit IRC | 02:40 | |
| *** imacdonn has quit IRC | 02:53 | |
| clarkb | dansmith: mriedem_afk fyi http://logs.openstack.org/21/620121/1/gate/nova-tox-functional-py35/bc7257e/job-output.txt.gz#_2018-12-08_01_56_29_026597 reset the gate | 02:54 |
| *** imacdonn has joined #openstack-infra | 02:54 | |
| *** bobh has joined #openstack-infra | 02:54 | |
| *** bobh has quit IRC | 02:58 | |
| *** bobh has joined #openstack-infra | 03:27 | |
| *** bhavikdbavishi has joined #openstack-infra | 03:29 | |
| *** bobh has quit IRC | 03:31 | |
| *** wolverineav has joined #openstack-infra | 03:40 | |
| *** wolverineav has quit IRC | 03:47 | |
| *** jamesdenton has quit IRC | 03:50 | |
| *** HenryG has quit IRC | 03:51 | |
| *** yamamoto has quit IRC | 03:52 | |
| *** bobh has joined #openstack-infra | 03:53 | |
| *** HenryG has joined #openstack-infra | 03:54 | |
| *** bobh has quit IRC | 03:58 | |
| *** mriedem_afk has quit IRC | 04:00 | |
| *** bobh has joined #openstack-infra | 04:09 | |
| *** jamesmcarthur has joined #openstack-infra | 04:12 | |
| *** bobh has quit IRC | 04:14 | |
| *** jamesmcarthur has quit IRC | 04:16 | |
| *** rh-jelabarre has quit IRC | 04:29 | |
| *** jamesmcarthur has joined #openstack-infra | 04:36 | |
| *** bobh has joined #openstack-infra | 04:37 | |
| *** bobh has quit IRC | 04:42 | |
| *** bobh has joined #openstack-infra | 04:50 | |
| *** bobh has quit IRC | 04:55 | |
| *** bhavikdbavishi has quit IRC | 05:00 | |
| *** yamamoto has joined #openstack-infra | 05:04 | |
| *** bobh has joined #openstack-infra | 05:05 | |
| *** bobh has quit IRC | 05:10 | |
| *** hwoarang has quit IRC | 05:12 | |
| *** hwoarang has joined #openstack-infra | 05:14 | |
| *** bobh has joined #openstack-infra | 05:24 | |
| *** bobh has quit IRC | 05:28 | |
| *** wolverineav has joined #openstack-infra | 05:29 | |
| *** wolverineav has quit IRC | 05:33 | |
| *** jamesmcarthur has quit IRC | 05:36 | |
| *** EvilienM is now known as EmilienM | 05:41 | |
| *** bobh has joined #openstack-infra | 05:51 | |
| *** bobh has quit IRC | 05:56 | |
| *** yamamoto has quit IRC | 06:00 | |
| *** bhavikdbavishi has joined #openstack-infra | 06:06 | |
| *** yamamoto has joined #openstack-infra | 06:07 | |
| *** bhavikdbavishi has quit IRC | 06:12 | |
| *** bobh has joined #openstack-infra | 06:18 | |
| *** bobh has quit IRC | 06:23 | |
| *** Douhet has quit IRC | 06:33 | |
| *** Douhet has joined #openstack-infra | 06:33 | |
| *** bobh has joined #openstack-infra | 06:34 | |
| *** bobh has quit IRC | 06:39 | |
| *** yamamoto has quit IRC | 06:40 | |
| *** yamamoto has joined #openstack-infra | 06:44 | |
| *** bobh has joined #openstack-infra | 06:48 | |
| *** bobh has quit IRC | 06:52 | |
| *** apetrich has joined #openstack-infra | 06:52 | |
| *** irdr has joined #openstack-infra | 06:54 | |
| *** bobh has joined #openstack-infra | 07:00 | |
| *** bobh has quit IRC | 07:05 | |
| *** ykarel|away has joined #openstack-infra | 07:06 | |
| *** yamamoto has quit IRC | 07:08 | |
| *** yamamoto has joined #openstack-infra | 07:16 | |
| *** yamamoto has quit IRC | 07:16 | |
| *** bobh has joined #openstack-infra | 07:26 | |
| *** bobh has quit IRC | 07:31 | |
| *** ykarel|away has quit IRC | 07:32 | |
| *** bobh has joined #openstack-infra | 07:39 | |
| *** bobh has quit IRC | 07:44 | |
| *** yamamoto has joined #openstack-infra | 07:52 | |
| *** bobh has joined #openstack-infra | 07:53 | |
| *** bobh has quit IRC | 07:58 | |
| *** bobh has joined #openstack-infra | 08:07 | |
| *** frickler_ is now known as frickler | 08:10 | |
| *** bobh has quit IRC | 08:12 | |
| *** bobh has joined #openstack-infra | 08:20 | |
| *** bobh has quit IRC | 08:24 | |
| *** yamamoto has quit IRC | 08:26 | |
| *** yamamoto has joined #openstack-infra | 08:26 | |
| *** jtomasek has joined #openstack-infra | 08:27 | |
| *** bobh has joined #openstack-infra | 08:32 | |
| *** yamamoto has quit IRC | 08:34 | |
| *** bobh has quit IRC | 08:37 | |
| *** bobh has joined #openstack-infra | 08:47 | |
| *** wolverineav has joined #openstack-infra | 08:48 | |
| *** bobh has quit IRC | 08:51 | |
| *** wolverineav has quit IRC | 08:52 | |
| *** bobh has joined #openstack-infra | 09:04 | |
| *** apetrich has quit IRC | 09:07 | |
| *** yamamoto has joined #openstack-infra | 09:07 | |
| *** bobh has quit IRC | 09:08 | |
| *** bobh has joined #openstack-infra | 09:17 | |
| *** bobh has quit IRC | 09:21 | |
| *** ykarel has joined #openstack-infra | 09:22 | |
| *** bobh has joined #openstack-infra | 09:29 | |
| *** bobh has quit IRC | 09:33 | |
| *** bobh has joined #openstack-infra | 09:42 | |
| *** bobh has quit IRC | 09:46 | |
| *** bobh has joined #openstack-infra | 09:59 | |
| *** ykarel has quit IRC | 10:01 | |
| *** bobh has quit IRC | 10:04 | |
| *** bobh has joined #openstack-infra | 10:13 | |
| *** tosky has joined #openstack-infra | 10:16 | |
| *** bobh has quit IRC | 10:18 | |
| *** ralonsoh has joined #openstack-infra | 10:19 | |
| *** bobh has joined #openstack-infra | 10:26 | |
| *** ralonsoh has quit IRC | 10:28 | |
| *** bhavikdbavishi has joined #openstack-infra | 10:50 | |
| *** yamamoto has quit IRC | 11:03 | |
| *** yamamoto has joined #openstack-infra | 11:04 | |
| *** takamatsu has quit IRC | 11:34 | |
| *** pfallenop has quit IRC | 11:45 | |
| *** ahosam has joined #openstack-infra | 12:02 | |
| *** pfallenop has joined #openstack-infra | 12:04 | |
| *** yamamoto has quit IRC | 12:21 | |
| *** ssbarnea|rover has joined #openstack-infra | 12:22 | |
| *** jamesdenton has joined #openstack-infra | 12:45 | |
| *** ykarel has joined #openstack-infra | 12:46 | |
| *** yamamoto has joined #openstack-infra | 12:59 | |
| *** ahosam has quit IRC | 13:11 | |
| *** ahosam has joined #openstack-infra | 13:12 | |
| fungi | sean-k-mooney: responding to a comment you made last night, there's been no nested virt in ovh for a while (i think it was crashing hypervisors?). they're working on trying to reenable it now that they've about finished upgrading to newton though | 13:20 |
| *** ahosam has quit IRC | 13:23 | |
| *** tosky has quit IRC | 13:23 | |
| fungi | at least that's what i recall them saying in the past day or two | 13:23 |
| *** ykarel has quit IRC | 13:27 | |
| *** elod has quit IRC | 13:41 | |
| *** elod has joined #openstack-infra | 13:43 | |
| *** bhavikdbavishi has quit IRC | 13:46 | |
| *** yamamoto has quit IRC | 14:11 | |
| *** mriedem has joined #openstack-infra | 14:16 | |
| mriedem | clarkb: ack on http://logs.openstack.org/21/620121/1/gate/nova-tox-functional-py35/bc7257e/job-output.txt.gz#_2018-12-08_01_56_29_026947 - we skipped that on master but not stable, i can propose a backport of the skip; thanks for pointing it out | 14:17 |
| *** apetrich has joined #openstack-infra | 14:26 | |
| *** efried has joined #openstack-infra | 14:37 | |
| *** efried_cya_jan has quit IRC | 14:37 | |
| *** slaweq has joined #openstack-infra | 14:39 | |
| *** efried is now known as efried_cya_jan | 14:46 | |
| *** ykarel has joined #openstack-infra | 14:51 | |
| *** bobh has quit IRC | 14:57 | |
| *** bobh_ has joined #openstack-infra | 14:57 | |
| *** dklyle has quit IRC | 15:49 | |
| *** beagles has quit IRC | 15:53 | |
| sean-k-mooney | fungi: ah ok good to know | 15:55 |
| *** slaweq has quit IRC | 16:03 | |
| *** jtomasek has quit IRC | 16:09 | |
| *** bobh_ has quit IRC | 16:15 | |
| *** bobh has joined #openstack-infra | 16:18 | |
| *** bobh has quit IRC | 16:45 | |
| *** bobh has joined #openstack-infra | 16:46 | |
| *** jamesmcarthur has joined #openstack-infra | 16:58 | |
| *** jamesmcarthur has quit IRC | 17:04 | |
| *** ykarel has quit IRC | 17:30 | |
| *** _alastor_ has quit IRC | 17:33 | |
| mriedem | this is probably not new, but looks like an easy fix https://bugs.launchpad.net/grenade/+bug/1807520 | 17:58 |
| openstack | Launchpad bug 1807520 in grenade "volume attach fails on stein side of grenade with: Volume ab2390bc-54a2-454e-823d-01faf0a8fdc1 status must be available or downloading (HTTP 400)" [Undecided,Confirmed] | 17:58 |
| mriedem | this also looks new https://bugs.launchpad.net/glance/+bug/1807518 | 18:01 |
| openstack | Launchpad bug 1807518 in Glance "BasicOperationsImagesTest.test_update_image fails with 502 error "UnexpectedContentType"" [Medium,Confirmed] | 18:01 |
| *** yamamoto has joined #openstack-infra | 18:14 | |
| *** yamamoto has quit IRC | 18:14 | |
| *** yamamoto has joined #openstack-infra | 18:14 | |
| *** jamesmcarthur has joined #openstack-infra | 18:17 | |
| sean-k-mooney | mriedem: clarkb submited a patch to tweak the apache2 config that might resolve that | 18:19 |
| sean-k-mooney | mriedem: https://review.openstack.org/#/c/623597/2 | 18:19 |
| *** jtomasek has joined #openstack-infra | 18:20 | |
| mriedem | yeah looks like the same thing from the commit message | 18:21 |
| mriedem | +1 | 18:24 |
| mriedem | patch/gerrit bot must be dead | 18:27 |
| mriedem | but this is the fix for grenade https://review.openstack.org/623669 | 18:27 |
| sean-k-mooney | ya that makes sense | 18:32 |
| *** jamesmcarthur has quit IRC | 18:55 | |
| *** jtomasek has quit IRC | 18:58 | |
| *** dhill_ has quit IRC | 19:00 | |
| *** jamesmcarthur has joined #openstack-infra | 19:25 | |
| *** pots has quit IRC | 19:29 | |
| *** mriedem has quit IRC | 19:33 | |
| *** dklyle has joined #openstack-infra | 19:36 | |
| *** yamamoto has quit IRC | 20:15 | |
| *** yamamoto has joined #openstack-infra | 20:16 | |
| *** bobh has quit IRC | 20:18 | |
| *** yamamoto has quit IRC | 20:20 | |
| *** hwoarang has quit IRC | 20:26 | |
| *** hwoarang has joined #openstack-infra | 20:27 | |
| *** bobh has joined #openstack-infra | 20:41 | |
| *** Krenair has quit IRC | 20:58 | |
| *** eernst has joined #openstack-infra | 21:02 | |
| *** eernst has quit IRC | 21:08 | |
| *** markvoelker has joined #openstack-infra | 21:26 | |
| *** bobh has quit IRC | 21:26 | |
| *** Krenair has joined #openstack-infra | 21:27 | |
| *** eernst has joined #openstack-infra | 21:43 | |
| *** bobh has joined #openstack-infra | 21:43 | |
| *** bobh has quit IRC | 21:47 | |
| *** bobh has joined #openstack-infra | 22:07 | |
| *** bobh has quit IRC | 22:12 | |
| *** bobh has joined #openstack-infra | 22:25 | |
| *** bobh has quit IRC | 22:29 | |
| *** jamesmcarthur has quit IRC | 22:31 | |
| *** bobh has joined #openstack-infra | 22:44 | |
| *** bobh has quit IRC | 22:49 | |
| *** bobh has joined #openstack-infra | 23:02 | |
| *** bobh has quit IRC | 23:06 | |
| *** eernst has quit IRC | 23:08 | |
| *** dklyle has quit IRC | 23:11 | |
| *** jamesmcarthur has joined #openstack-infra | 23:23 | |
| *** bobh has joined #openstack-infra | 23:27 | |
| *** jamesmcarthur has quit IRC | 23:29 | |
| *** bobh has quit IRC | 23:31 | |
| *** bobh has joined #openstack-infra | 23:45 | |
| *** bobh has quit IRC | 23:46 | |
| *** bobh has joined #openstack-infra | 23:47 | |
| *** bobh has quit IRC | 23:48 | |
| *** jamesmcarthur has joined #openstack-infra | 23:53 | |
| *** bobh has joined #openstack-infra | 23:53 | |
| *** jamesmcarthur has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!