| *** gouthamr has joined #openstack-shade | 00:24 | |
| *** larainema has joined #openstack-shade | 01:42 | |
| *** gouthamr has quit IRC | 02:31 | |
| *** gouthamr has joined #openstack-shade | 03:12 | |
| *** gkadam has joined #openstack-shade | 04:09 | |
| *** gouthamr has quit IRC | 04:56 | |
| *** gkadam_ has joined #openstack-shade | 05:38 | |
| *** gkadam has quit IRC | 05:40 | |
| *** gkadam__ has joined #openstack-shade | 05:40 | |
| *** gkadam__ has quit IRC | 05:42 | |
| *** gkadam_ has quit IRC | 05:43 | |
| *** gkadam has joined #openstack-shade | 05:43 | |
| *** gkadam has quit IRC | 06:40 | |
| *** gkadam has joined #openstack-shade | 06:42 | |
| *** slaweq has joined #openstack-shade | 07:01 | |
| *** ioggstream has joined #openstack-shade | 07:15 | |
| *** gkadam is now known as gkadam-afk | 07:24 | |
| *** gkadam-afk has quit IRC | 08:28 | |
| *** gkadam has joined #openstack-shade | 08:31 | |
| *** gkadam_ has joined #openstack-shade | 09:36 | |
| *** gkadam has quit IRC | 09:38 | |
| *** thingee has quit IRC | 10:28 | |
| *** gkadam_ has quit IRC | 12:46 | |
| *** gouthamr has joined #openstack-shade | 13:49 | |
| -openstackstatus- NOTICE: Job log uploads are failing due to lack of inodes. Jobs also fail due to mismatches in gnutls packages. Workarounds for both in progress with proper fixes to follow. | 14:55 | |
| *** ChanServ changes topic to "Job log uploads are failing due to lack of inodes. Jobs also fail due to mismatches in gnutls packages. Workarounds for both in progress with proper fixes to follow." | 14:55 | |
| Shrews | mordred: so, umm... haven't been in actual shade code for a while due to zuulv3 things, but, what's the deal with this test? http://git.openstack.org/cgit/openstack-infra/shade/tree/shade/tests/functional/test_image.py#n32 | 16:10 |
|---|---|---|
| Shrews | i guess the test is that it doesn't throw any exceptions. :/ | 16:11 |
| Shrews | also, http://git.openstack.org/cgit/openstack-infra/shade/tree/shade/openstackcloud.py#n4719 seems to be broken when using tasks for image upload because status is a list in that case. our unit test only assumes a dict | 16:12 |
| Shrews | mordred: i have NO idea how any images are being uploaded to rax, atm, because image uploads are continually failing | 16:12 |
| Shrews | mordred: also, what I find REALLY confusing, after looking at nodepool builder logs, is that we attempt to upload for hours until it finally succeeds. The fact that it ever succeeds (meaning that error is not encountered somehow?) is amazing and perplexing | 16:25 |
| Shrews | pabelanger: ^^^ likely the reason why we see slooooooow uploads | 16:26 |
| Shrews | mordred: so do we need to check if 'status' in the 'status.status' check is a list? I'm a bit confused on what's being returned from the REST call | 16:27 |
| Shrews | is the /tasks/{id} return sometimes a list, sometimes a dict? | 16:28 |
| mordred | Shrews: hrm. that's an interesting question ... and I agree, I'm not sure how that is working - or honestly how it's passing unit tests | 16:38 |
| mordred | Shrews: oh - I think I see how it's passing unit tests ... line 339 in shade/tests/unit/test_image.py | 16:39 |
| mordred | sorry 342 | 16:39 |
| Shrews | mordred: http://paste.openstack.org/show/623475/ | 16:39 |
| mordred | yah | 16:39 |
| Shrews | mordred: yeah, our 'args' is a dict | 16:39 |
| Shrews | we don't have a test for the [args] case | 16:40 |
| mordred | well, that should never be a thing | 16:40 |
| mordred | status = self._get_and_munchify('images', data=data) | 16:40 |
| mordred | is what's wrong | 16:40 |
| Shrews | but that can return a list | 16:40 |
| Shrews | of munches | 16:40 |
| Shrews | if data is a list, it returns a list | 16:40 |
| mordred | right - but only bcause 'images' isn't a key | 16:40 |
| mordred | so result = data.get(key, []) if key else data | 16:40 |
| mordred | in get_and_munchify | 16:41 |
| mordred | is causing [] to be returned | 16:41 |
| mordred | https://developer.openstack.org/api-ref/image/v2/index.html#show-task-details | 16:41 |
| Shrews | ah | 16:41 |
| mordred | GET /tasks/{id} returns a bare dict | 16:41 |
| Shrews | oh look, they finally have task docs | 16:41 |
| SamYaple | haha | 16:42 |
| SamYaple | yea i was suprised that no one knew about tasks for a long time | 16:42 |
| SamYaple | i told #infra about tasks and no one had heard of them. threw me for a loop | 16:42 |
| mordred | Shrews: so I think the issue is that get_and_munchify call - it should be key=None - or honestly just meta.obj_to_munch(data) | 16:43 |
| mordred | Shrews: OR - we never return that status to anyone there - so we can skip the munch call altogether | 16:43 |
| Shrews | mordred: i wonder why it sometimes works? that bothers me | 16:45 |
| mordred | Shrews: yah - me too | 16:45 |
| mordred | Shrews: unless by "work" what we mean is that the *initial* attempt to import the image eventually works - and we notice that eventually with get_image calls on the retry of theupload | 16:46 |
| mordred | and all of our polling of task status is bong in the mean time and we create a bunch of duplicate task imports that all are bong | 16:46 |
| Shrews | possibly | 16:46 |
| mordred | but since we don't ever re-upload the object to swift - nor do we have a way to cancel the initial task | 16:46 |
| mordred | the initial task will eventually result in an image with the name we wanted | 16:47 |
| Shrews | mordred: so is the unit test wrong returning json keyed by 'images'? | 16:49 |
| mordred | Shrews: yup. so something like this: http://paste.openstack.org/show/623476/ perhaps | 16:50 |
| Shrews | mordred: i would readily approve such a change | 16:51 |
| mordred | Shrews: WELL - we may need to figure out why the shade gate is unhappy too | 16:51 |
| Shrews | ugh | 16:51 |
| mordred | Shrews: which I've been deferring until post-v3 rollout | 16:52 |
| Shrews | oh my. that's a lot of -1's :( | 16:52 |
| mordred | yah | 16:52 |
| mordred | Shrews: I think it may be related to that os_image error that snuck in to ansible upstream | 16:53 |
| mordred | Shrews: yah- I think that's the case - let me update the 'disable volume tests for now' patch to also disable os_image tests for now | 16:56 |
| mordred | Shrews: then we can get the image fix in, cut a release, and circle back on os_image and the volume deleting issue | 16:57 |
| Shrews | nod | 16:57 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Temporarily disable volume and os_image functional tests https://review.openstack.org/508156 | 17:15 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Fix image task uploads https://review.openstack.org/511532 | 17:15 |
| *** slaweq_ has joined #openstack-shade | 17:23 | |
| *** slaweq_ has quit IRC | 17:24 | |
| *** thrash is now known as thrash|biab | 17:45 | |
| *** slaweq_ has joined #openstack-shade | 17:55 | |
| *** ioggstream has quit IRC | 18:19 | |
| *** thrash|biab is now known as thrash | 18:21 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Fix image task uploads https://review.openstack.org/511532 | 18:38 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Add group parameter to create_server https://review.openstack.org/511305 | 18:38 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Image should be optional https://review.openstack.org/511299 | 18:42 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Add method to set bootable flag on volumes https://review.openstack.org/502479 | 18:42 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Allow domain_id for roles https://review.openstack.org/496992 | 18:42 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Move role normalization to normalize.py https://review.openstack.org/500170 | 18:42 |
| openstackgerrit | Merged openstack-infra/shade master: Temporarily disable volume and os_image functional tests https://review.openstack.org/508156 | 18:52 |
| *** slaweq_ has quit IRC | 18:59 | |
| *** slaweq_ has joined #openstack-shade | 19:03 | |
| *** slaweq_ has quit IRC | 19:06 | |
| *** slaweq_ has joined #openstack-shade | 19:06 | |
| *** gouthamr has quit IRC | 19:15 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade master: Image should be optional https://review.openstack.org/511299 | 19:34 |
| Shrews | mordred: ^^^ fixed the test failure for you. 'group' was getting defined by a for loop above | 19:34 |
| Shrews | oh dang. i fixed it on the wrong review | 19:35 |
| Shrews | *sigh* | 19:36 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade master: Add group parameter to create_server https://review.openstack.org/511305 | 19:37 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade master: Image should be optional https://review.openstack.org/511299 | 19:37 |
| Shrews | hrm, guess i could have fixed the 'payload' thing too. wanna make sure what you intended there, first | 19:39 |
| mordred | Shrews: thanks! and yes - s/payload/kwargs/ is totally correct | 19:44 |
| openstackgerrit | Merged openstack-infra/shade master: Fix image task uploads https://review.openstack.org/511532 | 20:04 |
| *** gouthamr has joined #openstack-shade | 20:16 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade master: Add method to set bootable flag on volumes https://review.openstack.org/502479 | 20:45 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade master: Add group parameter to create_server https://review.openstack.org/511305 | 20:47 |
| Shrews | mordred: k, fixed the var name ^^^ | 20:47 |
| mordred | Shrews: thank you | 20:47 |
| Shrews | almost approved things too fast there :/ | 20:48 |
| Shrews | it's dependents will need a rebase b/c i'm lazy | 20:49 |
| pabelanger | Shrews: mordred: http://paste.openstack.org/show/623499/ | 20:56 |
| pabelanger | figured I post here about shade errors I am seeing in nodepool | 20:57 |
| pabelanger | some reason, we have multiple matches on an image | 20:57 |
| mordred | pabelanger: that's likely because of the other issue | 20:58 |
| Shrews | pabelanger: i'm guess that may be due to that bug i found earlier | 20:58 |
| Shrews | yeah | 20:58 |
| pabelanger | kk | 20:58 |
| pabelanger | I'll try to manually delete | 20:58 |
| pabelanger | or atleast see what images are there | 20:58 |
| Shrews | pabelanger: they might be stacked up. curious to know what you find | 20:59 |
| pabelanger | Shrews: mordred: we have indeed leaked some uploads: https://pastebin.com/raw/2LwNvAEg | 21:05 |
| Shrews | wheeeeeee | 21:05 |
| pabelanger | once the fires are out, we should for loop openstack delete there | 21:06 |
| Shrews | mordred: fyi for us later, we should find out why brctl is not found in: http://logs.openstack.org/05/511305/4/check/gate-dsvm-nodepool-src-old-nv/ac27af1/console.html#_2017-10-12_20_57_25_201123 | 21:06 |
| mordred | Shrews: that just means the script failed before it got far enough to install brctl | 21:06 |
| mordred | pabelanger: we can write a quick little shade script for that :) | 21:07 |
| Shrews | oh, the real error is above | 21:07 |
| Shrews | ERROR: branch not allowed by features matrix: 0.2.0 | 21:07 |
| pabelanger | mordred: yah | 21:07 |
| *** gouthamr has quit IRC | 21:13 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade master: Image should be optional https://review.openstack.org/511299 | 21:40 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade master: Add method to set bootable flag on volumes https://review.openstack.org/502479 | 21:40 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade master: Allow domain_id for roles https://review.openstack.org/496992 | 21:41 |
| Shrews | mordred: the server group change is almost through gate. rebased the others | 21:42 |
| Shrews | mordred: going to let you poke at https://review.openstack.org/496992 since you had comments about it before | 21:42 |
| Shrews | but it lgtm | 21:43 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade master: Move role normalization to normalize.py https://review.openstack.org/500170 | 21:43 |
| mordred | Shrews: thank you for shepherding those | 21:44 |
| Shrews | np | 21:44 |
| *** slaweq_ has quit IRC | 21:50 | |
| *** gouthamr has joined #openstack-shade | 21:55 | |
| openstackgerrit | Merged openstack-infra/shade master: Add group parameter to create_server https://review.openstack.org/511305 | 22:10 |
| *** gouthamr has quit IRC | 22:30 | |
| *** ChanServ changes topic to "#openstack-shade" | 23:04 | |
| -openstackstatus- NOTICE: Workarounds are in place for libcurl and similar dependency errors due to stale ubuntu mirroring, and for POST_FAILURE results stemming from runaway inode utilization on the logs site; feel free to recheck failing changes for either of these problems now | 23:04 | |
| openstackgerrit | Merged openstack-infra/shade master: Image should be optional https://review.openstack.org/511299 | 23:06 |
| openstackgerrit | Merged openstack-infra/shade master: Add method to set bootable flag on volumes https://review.openstack.org/502479 | 23:07 |
| openstackgerrit | Merged openstack-infra/shade master: Allow domain_id for roles https://review.openstack.org/496992 | 23:07 |
| openstackgerrit | Merged openstack-infra/shade master: Move role normalization to normalize.py https://review.openstack.org/500170 | 23:07 |
| SamYaple | mordred: can we expect a 1.24.0 soon? | 23:24 |
| mordred | SamYaple: working on releasing it right now | 23:32 |
| mordred | SamYaple: might be morning before it exists | 23:32 |
| *** gouthamr has joined #openstack-shade | 23:46 | |
| SamYaple | no problem. i wont start to work on pulling it in until monday | 23:47 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!