Friday, 2019-11-01

*** goldyfruit_ has joined #openstack-sdks00:20
*** goldyfruit_ has quit IRC00:55
*** jkulik has quit IRC00:58
*** jkulik has joined #openstack-sdks00:59
*** dave-mccowan has quit IRC01:04
*** enriquetaso has joined #openstack-sdks01:46
*** goldyfruit_ has joined #openstack-sdks02:17
*** mtreinish has quit IRC03:28
*** enriquetaso has quit IRC03:33
*** dave-mccowan has joined #openstack-sdks03:37
*** goldyfruit_ has quit IRC03:40
*** dave-mccowan has quit IRC03:40
*** factor has joined #openstack-sdks03:47
openstackgerritgugug proposed openstack/openstacksdk master: tox: Keeping going with docs
openstackgerritgugug proposed openstack/openstacksdk master: tox: Keeping going with docs
*** gtema has joined #openstack-sdks08:45
*** jpich has joined #openstack-sdks08:52
*** ralonsoh has joined #openstack-sdks09:07
*** Dinesh_Bhor has quit IRC09:07
*** cdent has joined #openstack-sdks09:16
openstackgerritPierre Prinetti proposed openstack/python-openstackclient master: versions: Fix 'versions show' help message
*** gtema has quit IRC10:29
*** cdent has quit IRC11:28
*** cdent has joined #openstack-sdks11:44
*** gtema has joined #openstack-sdks12:11
*** mriedem has joined #openstack-sdks13:08
*** goldyfruit_ has joined #openstack-sdks13:15
*** enriquetaso has joined #openstack-sdks13:43
*** gtema has quit IRC13:43
*** gtema has joined #openstack-sdks13:44
*** gtema has quit IRC13:49
*** gtema has joined #openstack-sdks13:50
*** gtema has quit IRC13:55
*** gtema has joined #openstack-sdks14:17
umbSublimeIs there any way to query the placement API with the SDK ? I can't seem to figure it out14:28
fricklerumbSublime: iirc there's a plugin for that14:28
*** mtreinish has joined #openstack-sdks14:28
umbSublimeosc-placement ?14:28
fricklerumbSublime: exactly14:29
umbSublimeOk, that works ! thanks14:29
umbSublimefrickler: I don't think I've ever user anything but the sdk or requests to query the API, is there a way I can just register a proxy on my connection object from this ?14:32
fricklerumbSublime: oh, you said sdk and not osc, not sure about that. maybe cdent or others can help, but likely many are on their way to the summit currently14:35
umbSublimehehe no worries I have workarounds currently but it feels hacky :)14:35
efriedumbSublime: In the sdk you can create a raw placement proxy (connection.placement) and use the REST primitives (.get(), .put(), etc.)14:41
efriedExtensive examples in nova's report client...14:42
efried is where we create the proxy. That calls into...14:43
efriedwhich in nova's case uses oslo.config opts to create the connection; but you should use your regular clouds.yaml thing.14:44
umbSublimeefried: awesome thanks !14:44
efriedfor your purposes14:45
efriedconn = openstack.connection.Connection(...)14:45
efriedplacement_proxy = conn.placement14:45
efriedplacement_proxy.get(...) etc.14:45
fricklerefried: hmm, I'm assuming that that code is in nova for historical reasons, wouldn't it make sense to move it into a library? not sure whether osc-placement would be the right target or something like python-placementclient14:48
efriedfrickler: not sure how much history you're signing up for. Grab a cup of coffee and scoot closer to the fire...14:48
umbSublimeefried: I've tried what you suggest but I get this message "Skipping over invalid version data. No stability status in version."14:49
efriedumbSublime: what release are you on?14:49
umbSublimefor this specific cloud 'queens'14:50
*** jangutter has quit IRC14:50
efriedfrickler: The SchedulerReportClient previously talked to placement through a generic ksa Adapter. We cut over to the SDK in Train, after putting a bit of framework in the SDK to accept oslo.config opts. Having done that, the change was a one-liner (plus test):
efriedumbSublime: in queens placement was still part of nova, and sdk wasn't ready to create a generic placement proxy.14:51
umbSublimeAhh I see, no worries then, I'll move along with my work around and make a bit more generic14:51
efriedumbSublime: you *might* be able to get away with it if you upgrade your openstacksdk to train-level code. But it's also possible you'll encounter some incompatible changes.14:51
umbSublimemy sdk is at 0.3614:52
cdentpresumably the new sdk release will still work with old14:52
efriedyeah, I'm not so sure about that ^14:52
umbSublimein fact I was wondering which version to use. i couldn't find any documentatyion saying "Queens should use sdk version XX"14:53
cdentfrickler: one of the issues is that the previous PTL of placement (me) thinks people should just write get and put requests to placement as that's the way it was designed to be used, but the newer PTL (tetsuro) feels a bit differently, so there may be changes afoot. efried: meh. meh to that I say14:53
*** gtema has quit IRC14:53
cdentI would assume that mordred would be very sad if modern sdk can't talk to old clouds14:54
umbSublimecurrently with .36 I can use it on ocata deployment as well14:54
efriedfrickler: yes, what cdent said, I was getting to that. We will "never" have a python-placementclient. But we may someday add more SDK-ish methods in the SDK.14:54
efriedumbSublime: hm, I don't see anything from 0.36.0 to 0.37.0 that should be making a difference for you. It is possible that the hitch is placement being integrated in nova. A generic ksa adapter ought to be able to talk to $service with REST primitives; but depending which incarnation of the service-types-authority you're using...14:57
fricklerefried: cdent: o.k., that explains things, thx14:57
umbSublimeefried: I'm still new to all this, I'm not sure i get what you mean15:03
umbSublimeLike i can generate a ksa adapter and register my endpoint to it and just use REST methods on it ?15:04
efriedsorry, I didn't mean to leave that hanging umbSublime15:04
efriedyes, you should be able to do that.15:04
umbSublimeAnd would i be able to register that to my connection object ?15:05
efriedIf you're paying attention to requirements files, the fact that you have sdk 0.36.0 probably means you have os-service-types at a train-ish level, and that ost thinks that placement is no longer part of nova.15:05
umbSublimeOk that makes sense, and would explain why conn.placement doesn't work corrctly15:06
umbSublimeerr it works correctly just not for my deployment15:06
efriedumbSublime: I want to help you debug this, trying to think how best to do it.15:07
efriedDo you have the ability to set up an oslo.config style .conf file with ksa opts to talk to your placement service?15:07
umbSublimeis that the older 'export OS_VAR_NAME=bleh' conf format ?15:08
umbSublimereally though I'm able to talk to the placement API without issues as it is using requests module15:09
efriedcool, ksa is a wrapper around requests that handles the auth and service catalog pieces. And sdk is a wrapper around ksa that lets you use a yaml file for config.15:11
efried(sdk does much more than that obviously for other services, but not yet for placement)15:12
umbSublimeOhhh I see now, so I could just use ksa and at least get the same interface as conn.compute.<HTTP METHOD> instead of re-implementing it !15:12
efriedinstead of re-implementing the auth and service catalog pieces, yes15:13
efriedumbSublime: Here's an example of oslo.config style setup to talk to placement:
efried(from a random CI run on a random recent nova patch)15:14
efriedumbSublime: So if you wanted to go that route, you could15:17
efried- steal nova's get_ksa_adapter
efried- register those conf opts like this
efried- set up a .conf file like that one15:17
efriedand then you should be able to do:15:17
efriedadap = get_ksa_adapter('placement')15:17
efriedadap.get(...) etc.15:17
efriedI would be interested to know more about how the sdk is failing though. I'm not sure "Skipping over invalid version data. No stability status in version." should actually be preventing sdk from creating a proxy for you...15:19
efriedHowever, that ^ probably means that queens placement wasn't producing a good version document. cdent can you corroborate?15:20
umbSublimeI can paste the full stacktrace. It's something about parsing the endpoint15:20
cdentefried, umbSublime : it's likely it was not, at that time15:21
cdentbecause there was never any plan to have other versions, so for a while it was not a list, if I remember right15:21
cdentyeah, that15:21
umbSublimeIs there a version of the SDK where this would work though ? (prior to train release maybe?)15:22
cdentumbSublime: can we wind this back a bit more: what's your end goal?15:22
umbSublimeWe have some issues with allocations that don't exist15:22
umbSublimeI'm just making a little tool to help show and clean them15:23
cdentah, okay, yeah, that's a common thing, enough that in newer versions nova has some tooling for it15:23
umbSublimeI have all my code working right now, and the tool works. I'm just trying to make it more "standard" I guess15:23
cdenta good inclination but I'd say probably not worth it in this case15:24
cdentno "standard" was set with regard to placement around queens time15:24
umbSublimeagreed, it was also a bit of an exercise to get more familiar with the SDK15:24
efriedunfortunate that you tried with this particular combination :(15:25
efriedWe didn't have sdk talking to placement until mid-train, and iirc we had to make some tweaks to sdk to make that happen more smoothly.15:25
umbSublimeI appreciate all your help though. efried I think I'll give that technique you shared a try15:25
cdentumbSublime: you can run placement standalone, if you want something to talk to using modern techniques, for educational purposes15:26
umbSublimeOhh you mean on a devstack or something ?15:26
cdentno, much more standalone, let me find you a ref15:27
cdent as well as
umbSublimeOne day I'll try and get more involved into dev in openstack currently I'm just an operator trying to make my life easier :)15:27
cdentthat would be great, openstack badly needs more people with field experience working on it to make it easier to manage15:28
umbSublimeto be honest I was wondering is there is some type of portal where openstack operators (which don't necesseraly have all the dev experience) could share and discuss some future features/bugs etc.15:29
cdentthe openstack-discuss mailing list is supposed to be, in part, for that15:31
* umbSublime subscribed15:33
umbSublimeWhat is the actual goal the SDK is trying to fill (not historically, but nowadays) ? Is it meant to be a one stop shop for os services needing to talk to another ? Or is it more dev/ops oriented to standardize talking to all the different APIs15:34
* cdent defers to frickler and efried on that question15:35
mriedemumbSublime: #openstack-operators channel15:35
mriedemthough it's generally quiet15:35
efriedumbSublime: That's really a mordred (sitting shirtless on a mountain with supplicants bringing him burnt offerings) question.15:35
efriedbut I'll give it a shot...15:36
mriedemumbSublime: if you send to the ML, tag ops stuff with [ops] to filter properly in people's mailbox15:36
efriedYes, one stop shop. Originally I think for cloud operators writing tooling that talks to their cloud. Yes standardize how we talk to the different APIs.15:37
efriedOnly more recently have we started to want to use it as a way for one service to talk to another.15:37
umbSublimehence the merge of shade into the sdk in the bigining15:37
efriedWe made some inroads on that in nova in train: we're most of the way through replacing nova's use of python-ironicclient with sdk; and we already swapped out placement (which had been using ksa directly) as you saw.15:38
umbSublimeefried: One last question concerning using get_ksa_adapter. Here what is conf ? I know it's what you showed with the .ini style config. But what type of object do I use to generate it ?15:54
efriedumbSublime: it's something like:15:59
efriedfrom oslo_config import cfg15:59
efriedregister_ksa_options(..)  # This should register the opts onto cfg.CONF15:59
efriedcfg.CONF(args=['--conf-file', '/path/to/your.conf'])  # This loads up your config file into cfg.CONF15:59
efriedgotta run to a meeting, that ^ is a bit rough but hopefully gets you pointed in the right direction.15:59
umbSublimeefried: thanks that's exactly what I need :)16:00
*** mriedem is now known as mriedem_feeds16:16
openstackgerritPedro Henrique Pereira Martins proposed openstack/keystoneauth master: Fixes OIDC authentication with multiple IdPs
*** goldyfruit___ has joined #openstack-sdks16:45
*** goldyfruit_ has quit IRC16:47
*** cdent has quit IRC16:53
*** enriquetaso has quit IRC16:59
*** jpich has quit IRC17:18
openstackgerritMerged openstack/python-openstackclient master: versions: Fix 'versions show' help message
umbSublimeI must be missing something I don't see what could be a duplicate. I'm trying to dig in the code, but i'm a bit lost17:29
*** mriedem_feeds is now known as mriedem17:34
efriedumbSublime: You're just running this code in isolation?17:35
openstackgerritPierre Prinetti proposed openstack/python-openstackclient master: Fix 'versions show' helps & docs
efriedumbSublime: It looks right to me, but I was able to repro locally. Investigating...17:38
umbSublimeefried: yes, only the code in hastebin in a venv17:40
efriedumbSublime: oh, I see what's happening. I didn't notice you were importing from nova. That import is already registering the conf opts via nova.conf.__init__.17:43
efriedIf you want to be able to use this cleanly, I suggest just copying in the guts of register_ksa_options17:43 you don't need to rely on nova existing at all.17:44
umbSublimeOhh I see what you mean.17:44
efriedumbSublime: Um, except you can simplify a lot further17:45
efriedyou don't need to remove and re-add the _ADAPTER_VERSION_OPTS for your purposes.17:45
efriedthough I guess you can, shrug.17:45
umbSublimesame for get_ksa_adapter (I just use the guts of the function) ?17:46
openstackgerritPierre Prinetti proposed openstack/python-openstackclient master: Fix 'versions show' helps & docs
openstackgerritPierre Prinetti proposed openstack/python-openstackclient master: Fix 'versions show' helps & docs
umbSublimegot it working. Thanks again efried !18:04
*** ralonsoh has quit IRC18:06
umbSublimeHere is what I end up with
efriedumbSublime: Some of that stuff is nova-specific and/or legacy gorp you're dragging along. If this is for more than just playing around, I would suggest simplifying.18:15
umbSublimeefried: yup. I'm doing that right now :) but being unfamiliar with the codebase i'm taking my time :)18:16
efriedbut yeah, this looks good.18:17
efriedumbSublime: okay, let me know if you need help18:17
efriedfor example, if this is always going to be a placement-specific tool, you can remove all the stuff about service types and just hardcode 'placement'.18:17
efriedand as I said before, you probably don't care to scrub those version opts for your purposes.18:17
umbSublimeWell I'm baking this into my more general purpuse lib module, so I'll try and keep it generic18:17
umbSublimejust in case I need it for another service at some point18:18
umbSublimeefried: I tried without and I get the same stack-trace as when using the nova register_ksa_opts18:18
efriedyou would need to remove that hackup from both places18:18
efriedso 1) get_ksa_adapter_opts can go away and just use ks_loading.get_adapter_opts directly; and 2) kill L36-818:19
umbSublimeOhhh I see, remove lines for "Removing version opts" and the ones that "add dummy opts for version"18:20
efriedactually, it gets simpler.18:20
efriedI only did this conf.register(get_opts) thing because I had to hack out those version things.18:21
efriedThere ought to be a ks_loading method like ks_loading.register_adapter_conf_options which you can call directly.18:22
umbSublimeOk, I'll look into that18:22
efriedumbSublime: like this
umbSublimeFor some reason i'm getting "keystoneauth1.exceptions.catalog.EndpointNotFound: Could not find requested endpoint in Service Catalog." with that code (adapted to my env of course)18:37
efriedadd to your conf:18:39
efried service_type = placement18:39
efriedI simplified out the bit that detects that :P18:39
efriedhere's super pared down:
efriedbtw, raise_exc=False means that for 4xx/5xx responses, instead of raising an exception, you still just get a Response back, which you can inspect yourself.18:40
umbSublimethis fixed it for me
umbSublimeohh very neat trick !18:41
efriedand Response has a magic __bool__ so you can actually say things like:18:41
efriedif resp:18:41
efried   # status code was 2xx/3xx, do success path18:41
efried   # status code was 4xx/5xx, do failure stuff18:41
efriedumbSublime: yes, what fixed you there was the cfg.set_defaults(service_type='placement') which is the same thing as you setting it in the conf (but in code)18:42
efriedso, either way18:42
umbSublimeSay I want to wrap all this in a function that takes a region_name and return the adapter for the specified region. I'd need to do something similar to "example 3" and then do CONF(['--my-opt', "my_val"])
umbSublimeAhh even easier I can just override the region_name attribute from the adapter19:02
efrieddepending where in the flow you want to do it, you could do CONF.set_override(region_name='foo', group='placement')19:15
efried(I may not have that syntax exactly right)19:15
openstackgerritEric Fried proposed openstack/python-openstackclient master: neutron: autogenerate docs
openstackgerritEric Fried proposed openstack/python-openstackclient master: common: autogenerate docs
openstackgerritEric Fried proposed openstack/python-openstackclient master: Update a stale doc reference to use :neutron-doc:
openstackgerritEric Fried proposed openstack/python-openstackclient master: neutron: autogenerate docs
openstackgerritEric Fried proposed openstack/python-openstackclient master: Update a stale doc reference to use :neutron-doc:
openstackgerritEric Fried proposed openstack/python-openstackclient master: common: autogenerate docs
*** enriquetaso has joined #openstack-sdks20:30
*** slaweq has joined #openstack-sdks20:46
*** slaweq has quit IRC21:05
*** goldyfruit_ has joined #openstack-sdks21:08
*** goldyfruit___ has quit IRC21:10
*** enriquetaso has quit IRC21:10
*** goldyfruit___ has joined #openstack-sdks21:32
*** goldyfruit_ has quit IRC21:34
*** enriquetaso has joined #openstack-sdks21:34
*** enriquetaso has quit IRC21:35
*** mriedem has quit IRC21:39
*** goldyfruit___ has quit IRC22:51

Generated by 2.15.3 by Marius Gedminas - find it at!