Tuesday, 2019-08-06

*** mriedem has quit IRC00:38
*** altlogbot_0 has quit IRC01:37
*** altlogbot_1 has joined #openstack-sdks01:39
*** ricolin has joined #openstack-sdks02:16
*** gtema has joined #openstack-sdks02:30
*** gtema has quit IRC02:34
*** slaweq has quit IRC02:35
*** whoami-rajat has joined #openstack-sdks03:08
*** yolanda has quit IRC03:21
*** yolanda has joined #openstack-sdks03:22
*** Luzi has joined #openstack-sdks05:05
*** gkadam_ has joined #openstack-sdks05:54
*** gkadam_ has quit IRC05:54
*** gkadam has quit IRC05:56
*** slaweq has joined #openstack-sdks07:06
*** tosky has joined #openstack-sdks07:24
*** ralonsoh has joined #openstack-sdks07:43
*** cdent has joined #openstack-sdks07:44
*** jpich has joined #openstack-sdks07:51
*** gtema has joined #openstack-sdks07:56
*** gtema has quit IRC08:14
*** gtema has joined #openstack-sdks08:14
*** dtantsur|afk is now known as dtantsur08:22
*** gtema has quit IRC08:24
dtantsurShrews, mordred: there was an existing patch for the traits problem, but thanks for fixing it anyway08:24
*** e0ne has joined #openstack-sdks08:35
*** jpich has quit IRC09:38
*** jpich has joined #openstack-sdks09:39
openstackgerritMerged openstack/openstacksdk master: Update betacloud vendor entry  https://review.opendev.org/67471510:22
openstackgerritMerged openstack/openstacksdk master: Add Python 3 Train unit tests  https://review.opendev.org/66749410:22
dtantsurmordred: we don't support global_request_id in SDK, do we?10:32
dtantsuralso it seems that the Volume support it very basic, right?10:36
*** jpich has quit IRC11:41
*** jpich has joined #openstack-sdks11:42
*** zbr is now known as zbr|lunch12:01
mordreddtantsur: I'm not sure what global_request_id is? so probably not - but maybe we do via keystoneauth?12:10
mordreddtantsur: and yeah - the openstack.block_storage layer could use more love for sure.12:11
dtantsuroh, yeah, global_request_id is a keystoneauth thing. I just need to know how to pass it to a Connectin..12:25
*** zbr|lunch is now known as zbr12:38
mordreddtantsur: ok. I just read the global_request_id stuff in ksa12:44
mordredthis is going to be a fun one ... because it's supposed to be set per-request, but the individual requests aren't partcularly exposed12:45
mordreddtantsur: I almost think we should make a $something like a context manager or something, so you coudl say with conn.global_request_id(the_id) as req_conn: req_conn.baremetal.machines() ... or something similar12:46
mordred(because a Connection can be used from a multi-threaded context, so just setting a global_request_id attribute on the Connection object so that it picked it up and passed it in to the request arguments would be a step, but it wouldnt' be the full picture)12:48
dtantsurIronic currently creates a connection per request (may end up in 1-10 sdk requests, but still)12:48
dtantsurbut I like the idea with a context manager12:49
mordreddtantsur: let me poke for just a sec - I think I can make you a patch real quick12:50
*** mriedem has joined #openstack-sdks13:00
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Add support for global_request_id  https://review.opendev.org/67480713:04
mordreddtantsur: ^^ how about that?13:04
dtantsurlemme see13:04
mordreddtantsur: (it should also let you just set it in the constructor in ironic since you're only doing connection per request anyway)13:04
rm_workdtroyer: apparently here13:23
dtroyerrm_work:  +113:24
rm_workdtroyer: barbican meeting happening right now and there's some discussion of the right direction for client work, like ... python-Xclient, openstack-sdk, and interfacing with openstackclient etc13:25
rm_worki dunno if they want to spend meeting time on it or if we might just follow up after, but i know you had some thoughts on this from the last time we talked at the PTG I think?13:25
rm_workthey're considering deprecating their python-barbicanclient and moving to openstacksdk only? is that right right direction?13:27
rm_workand then would they write in openstack-client support via the SDK code?13:27
dtroyerrm_work: I'm in another meeting at the moment but everything is basically converging on SDK, so I'd support tha13:27
rm_worknp maybe can talk after13:27
mordredrm_work: yes - deprecating python-*client and moving to SDK is the right direction for sure13:31
*** olaph has quit IRC13:34
rm_workok cool thanks :)13:43
efriedmordred, dtantsur: I just added that global_request_id kwarg in ksa's Session.request so you can set it per call13:55
efriedI haven't looked at what you're doing in sdk yet...13:56
mordredefried: yah - I bumped the sdk min so we can consume it13:56
efriedah cool13:56
mordred(and added some hooks and stuff)13:56
*** Luzi has quit IRC13:56
efriedokay, I'll take a closer look.13:56
*** altlogbot_1 has quit IRC14:12
*** altlogbot_3 has joined #openstack-sdks14:14
Shrewsdtantsur: oops, sorry for missing the traits fix14:20
*** jpich has quit IRC15:35
*** jpich has joined #openstack-sdks15:36
*** ricolin has quit IRC16:20
*** tosky has quit IRC16:25
*** jpich has quit IRC16:37
Shrewsmordred: so that code... seems to delete a single object, yet from my cleanup, i see multiple objects for each image. does sdk split that up into separate objects, or is that something rax does behind the scenes?16:37
mordredShrews: it's something we do ... one sec16:38
mordredShrews: if you look in openstack/cloud/_object_store.py - and look for "use_slo"16:38
mordredShrews: and then _upload_large_object16:39
mordredShrews: https://docs.openstack.org/swift/latest/overview_large_objects.html is the docs for the feature16:40
Shrewsok. i'll have to study this16:42
mordredShrews: and in delete_object in openstack/cloud/_object_store.py - we check to see if it was a static-large-object. if not, we pass the multipart-manifest = delete param which shoudl cause deleting the dynamic-large-object to delete all the segments. and if it is a static large object deleting the manifest is _supposed_ to delete the segments *I think*16:42
mordredit's also possibel that with static large objects it's our responsibility to delete the segments16:42
mordredor with dynamic16:42
mordredI might be getting the boolean backwards16:42
mordredanyway - yes - please learn things - I'm probably wrong about many parts16:43
timburke?multipart-manifest=delete works on SLOs -- it does *not* work on DLOs16:48
*** e0ne has quit IRC16:48
*** cdent has quit IRC16:48
mordredtimburke: on DLOs, we're supposed to delete all of the segments ourselves, yes?16:50
*** dtantsur is now known as dtantsur|afk17:07
*** ralonsoh has quit IRC17:08
timburkemordred, yup, client's gotta manage the segments for DLO17:16
*** mgariepy has quit IRC17:25
mordredShrews: ^^ - ok, so on DLO we're almost certainly leaking segments18:14
mordredShrews: but I don't see anywhere in nodepool where we're setting use_slo to false so I'd expect we're using SLO18:14
Shrewsmordred: would that come from a clouds.yaml setting?18:15
Shrewsrackspace profile maybe?18:16
Shrewshrm, all i see is "image_api_use_tasks": true18:17
*** e0ne has joined #openstack-sdks18:17
mordredyeah - it's not a setting we set in clouds.yaml18:29
mordredShrews: so - a few things ...18:29
mordredShrews: I think it's entirely possible that we will leak swift objects any time sometihng _else_ goes wrong in an image upload that doesn't trigger that finally18:30
mordredalso - maybe we leaked a bunch of things a while back - I think there was a time when the default of use_slo was the opposite18:30
mordredShrews: maybe on use_tasks clouds we should have delete_image go and look for the object that would have been used and try to delete it?18:31
* mordred just thinking out loud18:31
*** e0ne has quit IRC18:31
Shrewsmordred: is the slo manifest an object within itself? where would i find it?18:34
Shrewswondering if i can view it from the openstack client. all i see are the segments in 'images'18:35
mordredShrews: it is an object18:40
mordredShrews: iirc, there should be a bunch of segment objects like foo/123456 - and then there should be a manifest object foo18:40
mordredwhich should contain a json document pointing to all of the segment objects18:41
*** jangutter has quit IRC18:42
Shrewsmordred: ah, maybe something...18:47
Shrewsopenstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: https://storage101.ord1.clouddrive.com/v1/MossoCloudFS_28a933f9-900d-40b8-9de3-deb24d213aa2/images/opensuse-15-1564920980?multipart-manifest=put, Bad Request18:47
Shrewsmordred: if we fail to save the manifest, i bet we don't (can't?) delete the segments18:48
Shrewsthat one happened 2 days ago, and is the only image segments i did not delete in ORD because it was too recent18:49
Shrewsmordred: so feeling like your first suspicion might be viable18:50
mordredShrews: ah - yeah18:51
mordredShrews: sounds like we should maybe retry the manifest put - and if it exceeds retries maybe unwind deleting teh segments since we do have the list of segments in the manifest at that point?18:52
Shrewsmordred: yup18:52
mordredShrews: also - a retry at that point is WAY cheaper than a hard-fail followed by re-uploading all of the segments later18:53
Shrewswill see what kind of code i can barf out after the meeting18:56
Shrewsmordred: also, i'm guessing the 'images_segments' container is an old artifact? doesn't seem to be used18:57
Shrewsit exists in rax, but empty across the board18:58
*** e0ne has joined #openstack-sdks19:04
*** mriedem has quit IRC19:08
mordredyeah - I thnk old19:08
*** mriedem has joined #openstack-sdks19:09
mordredefried: dammit. your sensible request for tests has caused me to remember that something else is broken and find a whole new thing is broken19:12
*** e0ne has quit IRC19:13
*** e0ne has joined #openstack-sdks19:14
*** e0ne has quit IRC19:15
openstackgerritMonty Taylor proposed openstack/openstacksdk master: Add support for global_request_id  https://review.opendev.org/67480719:17
mordredefried, dtantsur|afk: ^^ that adds a test and fixes the other issues - and also 2 things are borked that I would like to get fixed19:17
efriedmordred: ack, and #sorrynotsorry19:49
*** e0ne has joined #openstack-sdks19:51
mordredShrews: we're naming the swift object after the image name, right?20:00
Shrewsmordred: i believe so20:01
mordredShrews: so we could also do a scan of objects matching the pattern ${imagename}/${index} on image delete perhaps?20:01
Shrewsmordred: well, yeah, but why? as an extra precaution?20:02
efriedmordred: reviewed20:03
Shrewsmordred: unless in rax, the images automatically reference the swift object? i had assumed that there was some step after uploading to make it a proper "image", but maybe that's not correct20:05
mordredShrews: well - there's no swift object anywhere but rax20:07
mordredShrews: but yeah - extra precaution - probably not worth the energy20:07
*** slaweq has quit IRC20:09
*** factor has joined #openstack-sdks20:22
*** slaweq has joined #openstack-sdks20:25
*** slaweq has quit IRC20:30
*** mriedem is now known as mriedem_afk20:46
*** e0ne has quit IRC20:48
*** whoami-rajat has quit IRC20:56
*** mriedem_afk is now known as mriedem21:20
*** mriedem has quit IRC23:40

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!