| *** itlinux_ has quit IRC | 00:04 | |
| cmurphy | bug report for our doc build problem https://github.com/sphinx-doc/sphinx/issues/6447 | 00:25 |
|---|---|---|
| lifeless | kmalloc: yo | 01:02 |
| kmalloc | Hey | 01:03 |
| lifeless | kmalloc: from the discussion above, I'm fairly sure you're way off in the weeds yeah :) | 01:03 |
| kmalloc | Re testresources? | 01:04 |
| lifeless | yah | 01:04 |
| kmalloc | Somewhat | 01:04 |
| kmalloc | Be back in 10 mins | 01:06 |
| kmalloc | Can I pick your brain then? | 01:06 |
| lifeless | in the test you want resources = [(db, mydb)] and mixin ResourcedTestCase (or reimplement - see https://github.com/testing-cabal/testresources/blob/master/testresources/__init__.py#L797) | 01:07 |
| lifeless | yes, I will braindump here till then | 01:07 |
| lifeless | before that you want to say | 01:07 |
| lifeless | mydb = FixtureResource(SomeFixture()) | 01:08 |
| lifeless | where SomeFixture is a Fixture that when setup will have whatever expensive thing you need (I'm presuming its a database, it usually is) | 01:08 |
| lifeless | then in fixture.reset() you should have any code you need to optimise your state purging | 01:10 |
| lifeless | e.g. rolling back your transactions to a PIT marker or whatever trick you are using. | 01:10 |
| lifeless | the manager always returning dirty will cause reset to be called always which will cause the fixture to have control over cleaning always | 01:11 |
| lifeless | mydb can then be used in any number of test classes and testresources will smooth across them all | 01:13 |
| *** spsurya has joined #openstack-keystone | 01:19 | |
| clarkb | q | 01:25 |
| clarkb | (sorry) | 01:26 |
| lifeless | qq! :P | 01:31 |
| kmalloc | lifeless: ahh, so you need at least one module level instance of the fixture/resource to hold the use count at 1 | 01:31 |
| kmalloc | otherwise the .tearDown (regardless of dirtied) will cleanup current_resource | 01:32 |
| kmalloc | that is what i think we were missing. | 01:32 |
| lifeless | not quite | 01:32 |
| lifeless | FixtureResource is a resource *manager* | 01:32 |
| kmalloc | Right | 01:32 |
| lifeless | it will instantiate the resource and that will setup the fixture for you | 01:32 |
| lifeless | and track the use counts | 01:32 |
| kmalloc | the issue we were seeing was every single test case was still creating a new instance of the fixture | 01:33 |
| lifeless | the module level instance you have is just the instance of the manager | 01:33 |
| kmalloc | just like today, because i think we were hitting a use count of 0, and ._setResource(None) was called in .tearDown | 01:33 |
| lifeless | so the probable reason for that is a missing / misconfigured load_tests hook | 01:33 |
| kmalloc | very likely | 01:34 |
| lifeless | without the load_tests hook to wrap the OptimisingTestSuite around everything, each test is going to enter in with a use count of zero and setup just-in-time | 01:34 |
| kmalloc | yeah that's it then | 01:34 |
| kmalloc | lbragstad: ^ need load_tests hook | 01:34 |
| lifeless | but it sounds like you've also reimplemented make() and so on which if you're using fixtures you don't want to be doing, thats the older generation API | 01:34 |
| kmalloc | if we use standard fixtures we don't need .make() | 01:35 |
| kmalloc | ? | 01:35 |
| kmalloc | well. the bootstrapper isn't exactly a fixture | 01:35 |
| kmalloc | it leaned on a db handle (sqlite) but it does work. | 01:35 |
| lifeless | where's your patch | 01:35 |
| kmalloc | it sets some things up in a non-fixture way | 01:35 |
| kmalloc | it's mangled atm. | 01:35 |
| kmalloc | but | 01:35 |
| kmalloc | 'https://review.opendev.org/#/c/663065/1 | 01:36 |
| kmalloc | i see where it needs to go | 01:36 |
| kmalloc | based upon this conversation | 01:36 |
| kmalloc | and we should probably wrap the bootstrapping into a fixture (our db one, specifically) | 01:36 |
| kmalloc | or at least it's own one that leans on the db one | 01:36 |
| lifeless | so here is an example of the hook - | 01:38 |
| kmalloc | unfortunately I wont be able to iterate on that for lbragstad with travel and funeral this weekend. | 01:38 |
| lifeless | https://github.com/openstack/oslo.db/blob/master/oslo_db/sqlalchemy/test_fixtures.py#L619 | 01:38 |
| kmalloc | yeah i gathered that as soon as you said something ;) | 01:38 |
| lifeless | looks like its been turned into a HOF or two | 01:39 |
| lifeless | the doc strings there are sensible | 01:39 |
| lifeless | ok so, thats that side of it | 01:39 |
| kmalloc | i'll circle up with lbragstad tomorrow (if I can) and point him in this direction if i can't directly help iterate on it | 01:40 |
| kmalloc | this convo helped a lot | 01:40 |
| lifeless | cool | 01:40 |
| lifeless | I'm not sure if https://github.com/openstack/oslo.db/blob/master/oslo_db/sqlalchemy/test_fixtures.py#L445 is quite what I'd do myself | 01:40 |
| kmalloc | yeah, thats still a question | 01:41 |
| lifeless | but its kindof similar - and you can also see from that how FixtureResource sits in the system more clearly I think | 01:41 |
| kmalloc | yah | 01:41 |
| lifeless | perhaps the oslo.db devs weren | 01:42 |
| lifeless | 't aware of FixtureResource | 01:42 |
| kmalloc | i might take a stab at our key repo fixtures. | 01:42 |
| kmalloc | that looks way more straight forward | 01:42 |
| kmalloc | and more costly than a reflection sqlite DB with bootstrap | 01:42 |
| * kmalloc wonders if he can wire up the fernet repos to no actually dump crap on disk and just in-memory it | 01:43 | |
| kmalloc | with a fixture | 01:43 |
| lifeless | The point of OTS here starts to become clear when you have resources/fixtures that get optimised across multiple different test classes all for the same external resource (e.g. pg db instance) | 01:44 |
| lifeless | There's a postgresql adapter for testresources | 01:44 |
| kmalloc | our fernet key repos are that way | 01:44 |
| kmalloc | there is no reason to generate new crypto (entropy) data over and over and over and over and over | 01:45 |
| lifeless | tight | 01:45 |
| kmalloc | we do it a lot | 01:45 |
| lifeless | so you make a fixture to hold the entropy | 01:45 |
| lifeless | then you have a fixtureresource manager and put it somewhere | 01:45 |
| lifeless | then you just add ('entropy', entropy) to your resources list in any test class that needs it | 01:45 |
| lifeless | done | 01:46 |
| lifeless | no need to have them in one hierarchy | 01:46 |
| kmalloc | yup | 01:46 |
| kmalloc | well it dumps files on disk right now | 01:49 |
| kmalloc | so at the very least make that dump crap on disk happen once (except where we explicitly dirty the repo with a rotation) | 01:49 |
| kmalloc | in both cases (that and DB), it makes sense to use the testresources | 01:50 |
| *** dave-mccowan has quit IRC | 02:02 | |
| *** lbragstad has quit IRC | 02:14 | |
| *** itlinux has joined #openstack-keystone | 02:27 | |
| *** Dinesh_Bhor has quit IRC | 02:37 | |
| *** markvoelker has joined #openstack-keystone | 02:59 | |
| *** Dinesh_Bhor has joined #openstack-keystone | 03:19 | |
| *** markvoelker has quit IRC | 03:30 | |
| *** gyee has quit IRC | 04:02 | |
| *** threestrands has joined #openstack-keystone | 04:15 | |
| *** bnemec has quit IRC | 04:23 | |
| *** gmann has quit IRC | 04:23 | |
| *** bnemec has joined #openstack-keystone | 04:25 | |
| *** markvoelker has joined #openstack-keystone | 04:27 | |
| *** gmann has joined #openstack-keystone | 04:27 | |
| *** pcaruana has joined #openstack-keystone | 04:30 | |
| *** markvoelker has quit IRC | 05:00 | |
| *** tkajinam has quit IRC | 05:01 | |
| *** tkajinam has joined #openstack-keystone | 05:03 | |
| *** itlinux has quit IRC | 05:22 | |
| *** tkajinam has quit IRC | 05:33 | |
| *** markvoelker has joined #openstack-keystone | 05:56 | |
| *** tkajinam has joined #openstack-keystone | 06:03 | |
| openstackgerrit | zhaixiaojun proposed openstack/keystone master: Bump openstackdocstheme to 1.30.0 https://review.opendev.org/663243 | 06:19 |
| *** markvoelker has quit IRC | 06:28 | |
| *** takamatsu has joined #openstack-keystone | 06:32 | |
| *** rcernin has quit IRC | 06:59 | |
| *** aloga has quit IRC | 07:23 | |
| *** aloga has joined #openstack-keystone | 07:23 | |
| *** tesseract has joined #openstack-keystone | 07:25 | |
| *** do3meli has joined #openstack-keystone | 07:29 | |
| do3meli | hi - anyone aware on what happens to reviews where tox build failed due to unknown interpreted text role "paramref" from sqlalchemy package? f.e.: https://review.opendev.org/#/c/663063/ | 07:31 |
| *** whoami-rajat has joined #openstack-keystone | 07:45 | |
| *** xek has joined #openstack-keystone | 08:09 | |
| *** markvoelker has joined #openstack-keystone | 08:26 | |
| *** tkajinam has quit IRC | 08:37 | |
| *** threestrands has quit IRC | 08:39 | |
| *** openstackgerrit has quit IRC | 08:47 | |
| *** rcernin has joined #openstack-keystone | 08:57 | |
| *** markvoelker has quit IRC | 08:59 | |
| *** Emine has joined #openstack-keystone | 09:21 | |
| *** markvoelker has joined #openstack-keystone | 09:57 | |
| *** markvoelker has quit IRC | 10:29 | |
| *** stingrayza has quit IRC | 10:30 | |
| *** stingrayza has joined #openstack-keystone | 10:32 | |
| *** adriant has quit IRC | 11:12 | |
| *** markvoelker has joined #openstack-keystone | 11:26 | |
| *** raildo has joined #openstack-keystone | 11:42 | |
| *** xek has quit IRC | 11:49 | |
| *** xek has joined #openstack-keystone | 11:50 | |
| *** markvoelker has quit IRC | 12:00 | |
| *** gmann has quit IRC | 12:03 | |
| *** stingrayza has quit IRC | 12:06 | |
| *** stingrayza has joined #openstack-keystone | 12:08 | |
| *** adriant has joined #openstack-keystone | 12:34 | |
| *** rcernin has quit IRC | 12:37 | |
| cmurphy | do3meli: it's a known issue due to the latest sphinx release https://github.com/sphinx-doc/sphinx/issues/6447 working on a workaround | 12:42 |
| *** adriant has quit IRC | 12:43 | |
| *** adriant has joined #openstack-keystone | 12:46 | |
| do3meli | cmurphy: thanks for the heads up. will someone after the fix trigger new builds or does the owner of the review/PR needs to care? | 12:53 |
| *** lbragstad has joined #openstack-keystone | 13:17 | |
| *** ayoung has joined #openstack-keystone | 13:17 | |
| lbragstad | kmalloc lifeless interesting - reading back | 13:24 |
| *** rcernin has joined #openstack-keystone | 13:41 | |
| *** jamesmcarthur has joined #openstack-keystone | 13:48 | |
| *** spsurya has quit IRC | 13:55 | |
| lbragstad | so it sounds like we need to use the load_tests hook around all keystone tests? or just the ones we're optimizing with testresources? | 13:57 |
| *** jamesmcarthur has quit IRC | 14:09 | |
| lbragstad | also - it sounds like we should be using FixtureResource for bootstrapper? | 14:12 |
| lbragstad | bootstrapper was never a fixture before, it was just an object that did stuff during setUp | 14:13 |
| *** jaosorior has joined #openstack-keystone | 14:15 | |
| *** jamesmcarthur has joined #openstack-keystone | 14:18 | |
| *** joshualyle has joined #openstack-keystone | 14:19 | |
| *** jamesmcarthur has quit IRC | 14:22 | |
| *** bnemec has quit IRC | 14:23 | |
| cmurphy | do3meli: I'll retrigger all the failed jobs when it's fixed | 14:24 |
| *** bnemec has joined #openstack-keystone | 14:25 | |
| do3meli | thx | 14:26 |
| *** jamesmcarthur has joined #openstack-keystone | 14:26 | |
| *** itlinux has joined #openstack-keystone | 14:57 | |
| *** itlinux has quit IRC | 14:59 | |
| *** do3meli has quit IRC | 15:22 | |
| *** rcernin has quit IRC | 15:26 | |
| *** gyee has joined #openstack-keystone | 15:27 | |
| *** jaosorior has quit IRC | 15:27 | |
| *** itlinux has joined #openstack-keystone | 15:56 | |
| *** markvoelker has joined #openstack-keystone | 16:04 | |
| *** openstackgerrit has joined #openstack-keystone | 16:14 | |
| openstackgerrit | Colleen Murphy proposed openstack/keystone master: Pin Sphinx to <2.1.0 on py3 https://review.opendev.org/663373 | 16:14 |
| cmurphy | not sure if ^ will pass the requirements check but if it does it should unblock the gate | 16:14 |
| cmurphy | upstream sphinx seems open to fixing the problem but it doesn't sound like it will be easy | 16:15 |
| *** Emine has quit IRC | 16:30 | |
| *** xek has quit IRC | 16:37 | |
| *** bnemec has quit IRC | 16:44 | |
| *** itlinux has quit IRC | 16:56 | |
| *** itlinux has joined #openstack-keystone | 16:58 | |
| *** itlinux has quit IRC | 17:04 | |
| *** xek has joined #openstack-keystone | 17:05 | |
| *** xek has quit IRC | 17:08 | |
| *** xek has joined #openstack-keystone | 17:09 | |
| *** itlinux has joined #openstack-keystone | 17:09 | |
| gagehugo | ack | 17:15 |
| *** xek has quit IRC | 17:54 | |
| *** tesseract has quit IRC | 17:57 | |
| *** notq has joined #openstack-keystone | 18:27 | |
| notq | openstack ec2 credentials is not being logged by auditing/messaging, i've created a bug https://bugs.launchpad.net/keystonemiddleware/+bug/1831791 - should I add any other details, or is that suffient? | 18:27 |
| openstack | Launchpad bug 1831791 in keystonemiddleware "openstack ec2 credentials not audited" [Undecided,New] | 18:27 |
| *** itlinux has quit IRC | 18:32 | |
| kmalloc | lbragstad, cmurphy: so... for resource options for all, i am thinking of the following (some things can't be changed anyway) | 18:57 |
| kmalloc | users (done), groups, domains, projects, roles | 18:58 |
| kmalloc | the other question I have is should all options for all resources go in a single table? | 18:58 |
| kmalloc | so the key would be <type>, <option id> not just option_id | 18:59 |
| kmalloc | i am trying to avoid needing a db migration for each new resource we add options to | 19:00 |
| kmalloc | i think this is a low enough volume that a single, well indexed table for all options should be ok. | 19:00 |
| kmalloc | that way it really is just adding code support not code+migration | 19:01 |
| lbragstad | i can see the benefit of a single table if we expect an explosion of options to happen | 19:02 |
| *** itlinux has joined #openstack-keystone | 19:04 | |
| kmalloc | as it stands it would be a good chunk of code and migrations to just get the ones i've listed above, rather that make it a generic we can apply to the baseclasses | 19:06 |
| kmalloc | a single table simplifies that a lot | 19:08 |
| cmurphy | single table sounds okay to me, obviously don't use enum for type | 19:15 |
| cmurphy | notq: usually i'd recommend steps to reproduce, expected behavior and actual behavior | 19:17 |
| notq | cmurphy: fair, will add, thank you. | 19:18 |
| *** joshualyle has quit IRC | 19:18 | |
| notq | there's also a minor issue that openstack application credential create foo creates an audit message, but the target is unknown. unknown is given when it doesn't have a cadf translation for what is sent. Is there a way I can debug what is sent as target before it gets set to unknown, or do I have to add a statement to print it | 19:23 |
| *** joshualyle has joined #openstack-keystone | 19:28 | |
| lbragstad | cmurphy +1 to not using enum | 19:30 |
| cmurphy | notq: i don't have a good debugging recommendation but i think i may see the problem already | 19:31 |
| *** joshualyle has quit IRC | 19:32 | |
| notq | cmurphy: should i create another bug? want to point me at the problem? Just want to know next steps. | 19:33 |
| notq | O | 19:33 |
| notq | I should add, it happens for delete as well. | 19:35 |
| kmalloc | cmurphy: yah, it wont be a SQL enum for type, i'm going to use an attribute on the model. | 19:37 |
| kmalloc | lbragstad, cmurphy: the code will be: 1) migrate current options to the new single table | 19:39 |
| openstackgerrit | Colleen Murphy proposed openstack/keystone master: Add application_credential as a CADF type https://review.opendev.org/663410 | 19:39 |
| cmurphy | notq: ^ | 19:39 |
| kmalloc | 2) start adding new option code | 19:39 |
| notq | cmurphy++ | 19:41 |
| *** Emine has joined #openstack-keystone | 19:45 | |
| lbragstad | kmalloc makes sense | 19:45 |
| lbragstad | kmalloc when you're not traveling, we should sync up on the conversation you had with lifeless | 19:46 |
| lbragstad | or i'll try and be around later to catch him | 19:46 |
| *** imacdonn has quit IRC | 19:53 | |
| *** imacdonn has joined #openstack-keystone | 19:53 | |
| *** jamesmcarthur has quit IRC | 19:59 | |
| openstackgerrit | Colleen Murphy proposed openstack/keystone-specs master: Update access rules spec with decisions from PTG https://review.opendev.org/661784 | 20:15 |
| cmurphy | would like some feedback on that api change ^ | 20:16 |
| *** itlinux has quit IRC | 20:17 | |
| *** itlinux has joined #openstack-keystone | 20:19 | |
| *** joshualyle has joined #openstack-keystone | 20:44 | |
| *** joshualyle has quit IRC | 20:48 | |
| *** pcaruana has quit IRC | 20:59 | |
| *** Emine has quit IRC | 21:00 | |
| *** rcernin has joined #openstack-keystone | 21:26 | |
| openstackgerrit | Colleen Murphy proposed openstack/keystone master: Add user_id to access rules table https://review.opendev.org/663440 | 21:38 |
| openstackgerrit | Colleen Murphy proposed openstack/keystone master: Add flake8 ignore list to fast8 script https://review.opendev.org/663444 | 21:40 |
| *** gmann has joined #openstack-keystone | 21:45 | |
| *** dave-mccowan has joined #openstack-keystone | 21:46 | |
| *** itlinux has quit IRC | 21:50 | |
| *** itlinux has joined #openstack-keystone | 21:57 | |
| *** whoami-rajat has quit IRC | 22:00 | |
| *** adriant has quit IRC | 22:01 | |
| *** adriant has joined #openstack-keystone | 22:02 | |
| *** rcernin has quit IRC | 22:04 | |
| openstackgerrit | Colleen Murphy proposed openstack/keystone master: Stop treating Sphinx warnings as errors https://review.opendev.org/663373 | 22:05 |
| cmurphy | ^ second attempt | 22:05 |
| *** itlinux has quit IRC | 22:15 | |
| *** notq has quit IRC | 22:29 | |
| openstackgerrit | Colleen Murphy proposed openstack/keystone master: Add user_id to access rules table https://review.opendev.org/663440 | 22:37 |
| *** dave-mccowan has quit IRC | 22:56 | |
| *** raildo has quit IRC | 22:58 | |
| *** tkajinam has joined #openstack-keystone | 23:00 | |
| *** itlinux has joined #openstack-keystone | 23:16 | |
| *** itlinux has quit IRC | 23:46 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!