Tuesday, 2019-06-11

*** markvoelker has joined #openstack-sdks00:16
*** markvoelker has quit IRC00:49
*** ricolin has joined #openstack-sdks01:10
*** gkadam_ has quit IRC01:22
*** gkadam_ has joined #openstack-sdks01:22
*** gkadam_ has quit IRC01:38
*** gkadam_ has joined #openstack-sdks01:38
*** markvoelker has joined #openstack-sdks01:46
*** whoami-rajat has joined #openstack-sdks02:09
*** gkadam_ has quit IRC02:11
*** markvoelker has quit IRC02:20
*** markvoelker has joined #openstack-sdks03:17
*** gkadam has joined #openstack-sdks03:41
*** markvoelker has quit IRC03:50
*** gkadam has quit IRC04:11
*** dave-mccowan has quit IRC04:23
*** gkadam has joined #openstack-sdks04:28
*** gkadam has quit IRC04:35
*** markvoelker has joined #openstack-sdks04:47
*** e0ne has joined #openstack-sdks05:15
*** markvoelker has quit IRC05:21
*** e0ne has quit IRC05:42
*** Luzi has joined #openstack-sdks05:53
*** markvoelker has joined #openstack-sdks06:18
*** markvoelker has quit IRC06:51
*** slaweq has joined #openstack-sdks06:55
*** tosky has joined #openstack-sdks07:17
*** slaweq has quit IRC07:20
*** e0ne has joined #openstack-sdks07:20
*** dayou has quit IRC07:23
*** gtema has joined #openstack-sdks07:24
*** tssurya has joined #openstack-sdks07:29
*** dayou has joined #openstack-sdks07:37
*** jpena|off is now known as jpena07:46
*** markvoelker has joined #openstack-sdks07:48
*** ttsiouts has joined #openstack-sdks08:01
*** jpich has joined #openstack-sdks08:02
*** dtantsur|afk is now known as dtantsur08:02
*** holser_ has joined #openstack-sdks08:08
*** ralonsoh has joined #openstack-sdks08:09
*** holser_ has quit IRC08:10
*** holser_ has joined #openstack-sdks08:11
*** ttsiouts has quit IRC08:19
*** ttsiouts has joined #openstack-sdks08:20
*** markvoelker has quit IRC08:22
*** ttsiouts has quit IRC08:24
*** dayou has quit IRC08:28
*** ttsiouts has joined #openstack-sdks08:33
*** dayou has joined #openstack-sdks08:41
*** cdent has joined #openstack-sdks08:42
*** cdent has quit IRC09:00
*** ttsiouts has quit IRC09:06
*** ttsiouts has joined #openstack-sdks09:06
*** ttsiouts has quit IRC09:11
*** dayou has quit IRC09:18
*** ttsiouts has joined #openstack-sdks09:18
*** markvoelker has joined #openstack-sdks09:19
*** tobberydberg has quit IRC09:22
openstackgerritArtem Goncharov proposed openstack/openstacksdk master: Use resource layer for compute flavors  https://review.opendev.org/65090309:29
*** tobberydberg has joined #openstack-sdks09:31
*** dayou has joined #openstack-sdks09:31
openstackgerritArtem Goncharov proposed openstack/openstacksdk master: Use Resource layer for network SecurityGroups  https://review.opendev.org/66272409:35
*** cdent has joined #openstack-sdks09:36
*** gtema has quit IRC09:47
*** slaweq has joined #openstack-sdks09:47
*** markvoelker has quit IRC09:52
*** gtema has joined #openstack-sdks09:58
*** gtema has quit IRC10:28
*** ttsiouts has quit IRC10:28
*** gtema has joined #openstack-sdks10:28
*** ttsiouts has joined #openstack-sdks10:29
*** slaweq has quit IRC10:31
*** ttsiouts has quit IRC10:33
*** slaweq has joined #openstack-sdks10:39
*** markvoelker has joined #openstack-sdks10:49
*** slaweq has quit IRC10:59
*** markvoelker has quit IRC11:21
*** dave-mccowan has joined #openstack-sdks11:23
mordredgtema: morning! we ran in to https://storyboard.openstack.org/#!/story/2005851 with nodepool and had to pin to a previous version11:27
gtemamorning mordred11:27
mordredah - I see you commented on it ... yeah, that header is present11:27
gtemaIs it possible to see a real query going there?11:27
mordredI'm working on running some ... yeah - one sec11:27
mordredI'm working on doing the same upload again with 0.27 to get a comparable chunk of logging11:28
gtemahmm, looks good. Exactly11:28
mordredbecause I don't see anything wrong there - yeah11:28
mordredthe awesome part is that because rax uses the image-task method, I have to upload a full actually valid image :)11:32
*** jpena is now known as jpena|lunch11:38
gtemawhat I see could be different - presence of "Accept: "11:38
*** ttsiouts has joined #openstack-sdks11:40
mordredyes. although I did try a run hacking it to not send the accept header - and that sent the same 40011:43
gtemareally? hm11:44
gtemathat's the only thing I might think of11:44
mordredyeah. same here11:45
gtemabtw, to which version you downgraded?11:51
mordredgtema: 0.27.011:54
mordredgtema: which is what nodepool had been running before this recent restart that picked up latest sdk11:54
mordredok - so - I have confirmed the upload works under 0.27.011:54
gtemaand the request looks like ...?11:55
mordredthe main difference I can see in the REST calls is that in 0.27.0 that request is not made after the task import is complete - I THINK11:55
mordred(I really should have logged this to a file - scrollback is a bit hard to scane because of the list calls)11:55
mordredyou know what? I'm going to do it again and log to a file - then I can upload the whole file somewhere11:56
mordredhave I mentioned how happy I am that image uploading is so complicated? :)11:57
gtemanope, but I totally agree with you11:57
gtemaserver normalization (with respect to network addresses) is not much better11:58
gtemabut btw have found why nodepool was not happy with flavors - dogpile.cache is not happy with Resource, since it is returned from a function from a generator from a ....11:59
mordredI mean - ultimately a large percentage of the crazy logic in the shade layer is related to rackspace being weird (the one example in the other direction is how hard it is to boot a server with an ip address)11:59
mordredAH - joy11:59
mordredwell - I think we should rework how the caching works anyway ...11:59
gtemait's basically pickle.dumps(list(self.compute.flavors())) does not work11:59
gtemayeah, would need to11:59
mordredyou know what - maybe I should write a quick spec with the thoughts that were in my head WRT caching and we can talk it through and come up with an actual plan12:00
gtemabtw, racks support creating server with block_device_mapping2. Currently we use os-volumes_boot if bdm_v2 is present in params12:01
gtemafor caching - cool12:01
gtemalet's do this12:01
mordredso - I'm also thinking - the support you added for having a cloud/vendor-specific driver that overrides some of the proxies ...12:02
mordredmaybe it would be cleaner to extract the rackspace-specific logic we have into a rackspace driver12:02
*** gkadam has joined #openstack-sdks12:03
gtemathe ones "originated" all OpenStack are now a separate incompatible driver ???12:03
mordredyeah. it's "funny"12:04
mordredbut we carry a lot of logic that's just for them12:04
gtemabut yes, we can start really packing all of that into "drivers"12:04
mordredlike - the task import stuff - it's technically in upstream glance, but it's untestable in devstack12:04
*** dave-mccowan has quit IRC12:05
gtemaI'm back in 10-20 min12:06
gtemaam back12:18
mordredjust added the call from 0.27.012:19
gtemaso it's really just "Accept:"12:20
gtemawell and then the order of "path" and "value" for each item12:21
mordredI'm thinking maybe it's something to do with call sequencing12:22
mordredI'm going to re-run latest logging to a file12:23
gtemaif you quickly remove line 284 in image/v2/image.py (setting Accept) to be also sure12:23
mordredyeah - I did that yesterday12:23
mordredbut I can do it again and do the log if we want12:23
gtemawould be nice, but if you say it didn't help - not prio12:24
gtemalet's then really compare order or calls. However unittest tests this also. And here we had no change12:25
mordredyeah. I was pretty confident in the changes because we have this covered in unittests12:25
*** gkadam has quit IRC12:26
*** ricolin_ has joined #openstack-sdks12:27
*** gkadam has joined #openstack-sdks12:27
gtemalatest trace is broken - incomplete12:29
mordredoh yeah? poo - maybe there was a length limit here12:29
*** ricolin has quit IRC12:29
*** slaweq has joined #openstack-sdks12:29
mordredhttp://paste.openstack.org/show/752725/ <-- there's a trimmed version with just the request sequence12:30
gtemathe order is same as in tests12:31
mordredrunning 0.31.0 now - we'll see how it goes12:32
mordredtakes about 20 minutes12:32
*** jpena|lunch is now known as jpena12:43
*** zbr|ruck is now known as zbr|rover12:44
*** slaweq has quit IRC12:46
mordredthere's 0.31.0 sequence - looks the same :(12:49
*** ttsiouts has quit IRC12:49
gtemayeah. Is it now with "Accept: " or without?12:49
*** ttsiouts has joined #openstack-sdks12:50
gtemawhat I also see in the first trace before GET/PATCH image was first searched in the LIST. Could it "theoretically" be, that the ID changes?12:54
*** ttsiouts has quit IRC12:54
mordredwell - but in the failing one we do a GET on the image by id12:59
gtemayes , exactly13:01
mordred{"op": "add", "value": "False", "path": "/xenapi_use_agent"} 0.27.013:01
mordred{"op": "add", "path": "/xenapi_use_agent", "value": false} 0.31.013:02
mordredwe're passing false as a string in 0.27 and as a bool in 0.3113:02
gtemayeah, I see13:02
mordredin the original _make_v2_image_params - we made sure to str() wrap each item if it wasn't in a specific list13:05
mordredwe have the same thing in the image proxy13:06
mordredwe call it in master in _update_image_properties - but then we call self.update_image with the resulting dict - maybe that's flowing it back through the resource mapping and causing the False to get boolean'd13:07
mordredthat doesn't make any sense though - xenapi_use_agent isn't a known property - resource shouldn't be transforming its data type13:08
gtemayou know everything what's running in the background of all the methods ;-)?13:09
gtemashould we then force all the props before the line 277 (v2/_proxy) to update_image to be strings?13:12
gtemaor probably better in v2/image/_prepare_request (if patch ...)13:12
gtemawho is setting xenapi_ise_agent?13:13
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Filter properties through make_v2_image_params  https://review.opendev.org/66458213:15
mordredgtema: I *think* that ^^ should do the trick (looking at shade vs. current code, we're missing that call in that location)13:15
mordredgtema: but - the things you were saying are likely a *better* approach13:16
mordredgtema: xenapi_use_agent is actually set in the vendor profile of rackspace13:16
gtemayes, and I remember explicitely skipping it and relying to the general "patch" in the resource13:16
mordredoh. you know - it's set as boolean in the vendor file13:17
mordredmaybe let's try just updating that to "False"13:17
gtemathat's what I also mean. I think it will help either13:17
mordredI'm going to try patching that (the vendor profile) and trying another upload13:17
gtemaand probably it is a better place to fix. Normal patch is just comparing two things and generates a diff out of it.13:18
*** markvoelker has joined #openstack-sdks13:18
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Set xenapi_use_agent to "False" instead of false  https://review.opendev.org/66458513:20
*** ttsiouts has joined #openstack-sdks13:22
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Set xenapi_use_agent to "False" instead of false  https://review.opendev.org/66458513:22
*** markvoelker has quit IRC13:23
mordredalso - how is this just a raw 400 error13:23
gtemait's Glance13:23
mordredgood point13:23
gtemamordred: http://paste.openstack.org/show/752742/ - a current diff for using resource in cloud.server13:31
gtemawhat do you think about it?13:31
gtemabtw, if vendor fix works - nodepool should also get "aka" change included, since it will fix image checksum verification13:36
mordredI think that's looking good. it reminds me - I think as a followup, we should start using Flavor and SecurityGroup and Image resource objects in server.flavor, server.security_groups and server.image - and probably same for volume13:38
mordredmight be nicer than just a munch13:38
gtemayeah, but be aware of caching - it does not support Resource objects properly as of now13:38
mordredoh yeah - definitely a "for later" sort of thing13:39
gtemabut still here for the server - I need to do a "dummy" normalization to fix image and some other stuff13:39
mordredYES - the "False" patch fixed it13:40
gtemathat's great13:40
mordredit is.13:40
mordredit's absurd13:40
openstackgerritArtem Goncharov proposed openstack/openstacksdk master: WIP Use resource layer for compute.server cloud methods  https://review.opendev.org/66459313:40
mordredbut it's great13:40
gtemasure. That's why in SDK we try to fix things, so that end-user stuff should not cope with those things13:41
*** gkadam has quit IRC13:41
gtemathe patch above - it shows current _normalize_server to make it "more_or_less" comparable with current behavior13:42
*** gkadam has joined #openstack-sdks13:42
mordredShrews, dtantsur: if you have a sec, would you look at https://review.opendev.org/#/c/663846/ - we need to cut a point release with the rackspace fix and it would be nice to get the aka fix in to fix checksumming too13:42
dtantsurwill try to, a bit busy today though13:43
gtemadtantsur: those a "one-liners" ;-)13:43
gtemahehe, almost13:43
Shrewsmordred: which of those fixes is the rackspace fix?13:45
gtemanope - https://review.opendev.org/#/c/66458513:46
mordredyeah. 664585 is the one13:46
mordredShrews: you're really going to enjoy it13:46
Shrewsall of those "While we are here" things would be easier to review as separate reviews13:48
* Shrews woke up grumpy. apologies13:49
gtemasure, the thing is that exactly this fix found the issue13:49
Shrews access_ipv4 = resource.Body('accessIPv4', aka='accessIPv4')13:51
Shrewswhy is that aka necessary?13:51
gtemathat'S a wrong patch you look at. Here we want to return Resources in the cloud layer. However currently13:51
gtemathe cloud layer for servers return things with "bad names"13:52
gtemaand not to break compatibility for current users we need to expose attributes under old names13:52
*** markvoelker has joined #openstack-sdks13:53
gtemaso resource['accessIPv4'] returns something reasonable13:53
gtemaShrews: the most interesting part of the story here is currently exposed attribute 'adminPass', which we do not want to see in SDK as regular attribute13:54
*** slaweq has joined #openstack-sdks13:56
*** jpena is now known as jpena|afk14:14
*** slaweq has quit IRC14:28
*** dave-mccowan has joined #openstack-sdks14:29
Shrewsmordred: i think i groked enough of 846 to vote on it14:41
*** jpena|afk is now known as jpena14:51
*** ttsiouts has quit IRC14:57
*** ttsiouts has joined #openstack-sdks14:58
*** ttsiouts_ has joined #openstack-sdks15:00
*** ttsiouts has quit IRC15:01
*** slaweq has joined #openstack-sdks15:07
*** Luzi has quit IRC15:18
*** obre has joined #openstack-sdks15:20
obreAre the python-sdk compatible with multiple domains in keystonev3? I am struggeling to use the sdk to retrieve users.15:21
*** gkadam has quit IRC15:22
*** gkadam has joined #openstack-sdks15:22
obreIf I use an object returned from "openstack.connect" (an openstack.connection.Connection) called c i can get a user where I know the ID with "c.identity.find_user(userid)", but I cannot retrieve a user where I just know the username.15:23
obreHow are we supposed to search for a user using the name?15:23
obre"c.identity.find_user(username, domain_id = domain.id)" does not help me either.15:24
obreI simply get a None returned.15:25
obreRetrieving users from the default-domain works though.15:25
mordredobre: it should be compatible with multiple domains, althoguh I haven't done a lot of using that myself. perhaps mnaser or cmurphy knows more15:30
mnaserobre: did you enable logging to see what requests are happening?15:31
gtemaI was experiencing nice nasty issue, that you need to modify your connection to get domain_scoped token15:32
gtemaand basically there is no easy way to establish connection and state which scope you want to connect to15:33
gtemaThis leads to need of having multiple connection with/without user_project_name/id and selecting appropriate15:33
*** e0ne has quit IRC15:36
*** gkadam has quit IRC15:53
*** gkadam has joined #openstack-sdks15:53
*** ttsiouts_ has quit IRC16:04
*** ttsiouts has joined #openstack-sdks16:05
openstackgerritArtem Goncharov proposed openstack/openstacksdk master: WIP Use resource layer for compute.server cloud methods  https://review.opendev.org/66459316:06
*** cdent has quit IRC16:06
openstackgerritArtem Goncharov proposed openstack/openstacksdk master: Use resource layer for compute flavors  https://review.opendev.org/65090316:09
*** ttsiouts has quit IRC16:10
*** gtema has quit IRC16:22
*** jpich has quit IRC16:33
*** whoami-rajat has quit IRC16:38
*** slaweq has quit IRC16:49
*** dtantsur is now known as dtantsur|afk16:51
*** slaweq has joined #openstack-sdks16:52
openstackgerritMerged openstack/openstacksdk master: Set xenapi_use_agent to "False" instead of false  https://review.opendev.org/66458516:52
*** markvoelker has quit IRC16:53
*** slaweq has quit IRC16:56
*** gtema has joined #openstack-sdks17:01
*** tssurya has quit IRC17:04
*** whoami-rajat has joined #openstack-sdks17:05
openstackgerritMerged openstack/openstacksdk master: Add access alias (aka) for the resource attributes  https://review.opendev.org/66384617:06
*** jpena is now known as jpena|off17:17
*** dtruong has quit IRC17:24
*** tosky has quit IRC17:30
*** dtruong has joined #openstack-sdks17:50
*** ralonsoh has quit IRC17:53
*** holser_ has quit IRC17:55
*** gtema has quit IRC18:38
*** ricolin_ has quit IRC18:38
*** markvoelker has joined #openstack-sdks18:55
*** dtruong has quit IRC18:58
*** dtruong has joined #openstack-sdks18:59
mordredremote:   https://review.opendev.org/664671 Release 0.31.1 of openstacksdk19:02
*** holser_ has joined #openstack-sdks19:05
*** markvoelker has quit IRC19:29
*** holser_ has quit IRC19:34
*** efried has quit IRC20:01
*** gkadam has quit IRC20:04
*** gkadam has joined #openstack-sdks20:05
*** efried has joined #openstack-sdks20:10
*** markvoelker has joined #openstack-sdks20:26
efriedmordred: argh, I just saw that ^, *just* pushed a change to use 0.31.0. What was wrong with .0?20:26
efriedanything I'm going to care about?20:26
efriedmordred: Also, in case you care, I figured out why I had to bump the minimum for os-service-types to fix the lower-constraints job in nova for the "placement uses sdk" patch. If you care, read on:20:30
efriedWe were previously requiring a minimum ost from before placement extraction, so placement's "project" was still "nova", so we were looking for placement's adapter creds in the [nova] section rather than the [placement] section, but because we were only at 0.30.0 we weren't raising an error at that point, just using defaults.20:30
mordredefried: nothing you're goign to care about - it's a Rackspace public cloud profile issue20:38
mordredefried: ah - makes sense re: ost20:38
*** markvoelker has quit IRC20:59
efriedmordred: incidentally, it looks like everything related to clouds.yaml-from-nova is going to get punted for a while.21:00
mordredefried: ok.21:10
*** mriedem has joined #openstack-sdks21:54
mriedemdtroyer: mordred: can one of you abandon this now? https://review.opendev.org/#/c/460059/21:54
mriedemand https://review.opendev.org/#/c/627801/21:55
mriedemand https://review.opendev.org/#/c/589012/21:55
mriedemthose are all related to the same thing that i fixed already https://review.opendev.org/#/c/659382/21:55
*** markvoelker has joined #openstack-sdks21:56
dtroyermriedem: yes, I was planning to do that, just haven't gotten far enough down on the list21:57
*** holser_ has joined #openstack-sdks22:14
*** slaweq has joined #openstack-sdks22:22
*** markvoelker has quit IRC22:30
*** slaweq has quit IRC22:30
*** holser_ has quit IRC22:37
*** whoami-rajat has quit IRC22:55
*** mriedem has quit IRC23:06
*** markvoelker has joined #openstack-sdks23:26

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!