| *** dbpiv has joined #ara | 00:37 | |
| *** dbpiv has quit IRC | 01:23 | |
| *** dbpiv has joined #ara | 01:40 | |
| *** dbpiv has quit IRC | 02:18 | |
| *** jparrill has quit IRC | 05:28 | |
| *** gvincent has quit IRC | 05:48 | |
| *** gvincent has joined #ara | 05:48 | |
| *** zbr|flow is now known as zbr|ooo | 06:11 | |
| *** herald85 has joined #ara | 07:09 | |
| *** sshnaidm has quit IRC | 11:52 | |
| *** sshnaidm has joined #ara | 11:56 | |
| *** dbpiv has joined #ara | 12:18 | |
| *** Jmainguy has quit IRC | 13:44 | |
| *** Jmainguy has joined #ara | 13:48 | |
| dmsimard | apollo13: I looked a little bit more at the db router middleware things. Our use case doesn't match the expectation that all databases are configured before launching django :( | 14:10 |
|---|---|---|
| dmsimard | like a lot of examples are about setting a write master, read replicas, etc | 14:10 |
| apollo13 | yes, that's what I said :) | 14:10 |
| apollo13 | we'd need a custom db backend | 14:10 |
| *** Jmainguy has quit IRC | 14:11 | |
| dmsimard | apollo13: it's not that I didn't trust you, it's that I'm learning how things work :p | 14:12 |
| dmsimard | I'll attempt to capture this in a github issue | 14:16 |
| *** herald85 has quit IRC | 14:22 | |
| dmsimard | apollo13: https://github.com/ansible-community/ara/issues/57 | 14:53 |
| *** Jmainguy has joined #ara | 15:53 | |
| *** sshnaidm is now known as sshnaidm|off | 16:04 | |
| *** Jmainguy has quit IRC | 16:30 | |
| *** Jmainguy has joined #ara | 16:31 | |
| *** pabelanger has joined #ara | 17:34 | |
| dmsimard | pabelanger: I've been so far unable to reproduce the issue in both 0.x and 1.0 | 17:46 |
| dmsimard | 0.x has integration tests that can run locally with run_tests.sh -- tried renaming one of the integration test playbooks with some accents and it didn't trigger | 17:47 |
| dmsimard | running full ansible integration tests against 1.0 with increased verbosity to get the full callback trace, might give a hint | 17:51 |
| pabelanger | dmsimard: yah, I plan to collect the database and store it for our jobs, but not sure when it will happen again. This was the first time I seen that failure in about 2 months of using ara | 17:52 |
| *** etienne has joined #ara | 18:22 | |
| dmsimard | apollo13: for that database router thing, could we just override get_connection_params ? https://github.com/django/django/blob/1564e42ad397021093585147875a21dae1a3b3fc/django/db/backends/sqlite3/base.py#L166-L192 | 20:18 |
| dmsimard | and figure out the plumbing to get there | 20:19 |
| apollo13 | yes, that or get_connection directly | 20:19 |
| apollo13 | the question is, what aside from the database we need to patch to get the wanted functionality *gg* | 20:21 |
| apollo13 | do we have to multiplex the API to all those paths? | 20:21 |
| dmsimard | apollo13: no | 20:23 |
| dmsimard | just one database at a time but the location of the database is not known until the request comes in | 20:25 |
| apollo13 | yes, but what does ara serve then? | 20:25 |
| apollo13 | the database directly? | 20:25 |
| apollo13 | I mean if you GET that URL what do you get? | 20:25 |
| dmsimard | yeah, as if configured normally | 20:25 |
| dmsimard | would be the api server with data from that database | 20:26 |
| apollo13 | but the api server doesn't directly serve from one url | 20:26 |
| apollo13 | we have multiple endpoints | 20:26 |
| apollo13 | couldn't the database file locations and endpoints clash if we have multiple nesting levels? | 20:26 |
| dmsimard | for 0.x there is a webserver config that delegates everything under /ara-report/* to the wsgi app | 20:27 |
| dmsimard | define multiple nesting levels | 20:27 |
| apollo13 | hu? | 20:28 |
| dmsimard | for example openstack-ansible has different databases | 20:28 |
| apollo13 | assume you have two database locations | 20:28 |
| dmsimard | from zuul http://logs.openstack.org/78/658178/1/check/openstack-ansible-deploy-aio_lxc-debian-stable/ad6fc29/ara-report/ | 20:28 |
| dmsimard | from openstack-ansible: http://logs.openstack.org/78/658178/1/check/openstack-ansible-deploy-aio_lxc-debian-stable/ad6fc29/logs/ara-report/ | 20:29 |
| apollo13 | /ara-report/db1 and /ara-repor/db1/db2 | 20:29 |
| dmsimard | yeah there needs to be a convention | 20:29 |
| apollo13 | like what? limited to one path segment? | 20:29 |
| dmsimard | the convention in 0.x is there needs to be an ansible.sqlite file in the /ara-report/ directory | 20:29 |
| dmsimard | that's also why you can download the database by hitting http://logs.openstack.org/78/658178/1/check/openstack-ansible-deploy-aio_lxc-debian-stable/ad6fc29/ara-report/ansible.sqlite | 20:30 |
| dmsimard | let me pull up the webserver config, might make more sense | 20:30 |
| apollo13 | sure, but how do you know that ansible.sqlite should serve the sqlite file and is not an API endpoint? | 20:30 |
| apollo13 | or rather can you give me an example of how the URLs would look like | 20:30 |
| dmsimard | apollo13: http://paste.openstack.org/raw/752901/ | 20:31 |
| apollo13 | ie one ara server instance serving 2 databases | 20:31 |
| apollo13 | dmsimard: right, but on ara 0.x you have just one URL which serves the whole frontend, right? | 20:31 |
| dmsimard | meaning it replies with the webapp as opposed to an API ? | 20:32 |
| apollo13 | yeah | 20:32 |
| dmsimard | right -- in our context instead of the webapp it'd be the API | 20:33 |
| apollo13 | yeah but our API has then multiple suburls | 20:33 |
| apollo13 | /plays, /files etc… | 20:33 |
| dmsimard | right so you'd have http://logs.openstack.org/78/658178/1/check/openstack-ansible-deploy-aio_lxc-debian-stable/ad6fc29/ara-report/api/v1/playbooks?name=foo | 20:33 |
| apollo13 | at which point you have to make sure that we do not run into conflicts URL wise | 20:33 |
| apollo13 | so basically what you are saying is search the url for "ara-report", take everything before as fs-path and everything afterwards as endpoint path for the api? | 20:34 |
| dmsimard | apollo13: the webserver config delegates everything under /ara-report/ so this works: http://logs.openstack.org/78/658178/1/check/openstack-ansible-deploy-aio_lxc-debian-stable/ad6fc29/logs/ara-report/result/c29919e1-1bf7-4017-8e3f-0df9ec705ec3/ | 20:34 |
| apollo13 | right | 20:34 |
| apollo13 | looking at your webserver config though, what would happen if you have ara-report twice in the path :) | 20:35 |
| apollo13 | or if we ever wanted to make ara-report and endpoint of the api | 20:35 |
| dmsimard | if you put ara-report twice in the path that's wrong :p | 20:35 |
| apollo13 | that said, if you don't care about all that we might be able to handle it | 20:35 |
| dmsimard | the scope needs to be narrow -- at least to begin with | 20:37 |
| apollo13 | ok, so all we would need to do compared to the current setup is point to another settings file which has another ROOT_URLCONF which basically does include('<string>/ara-report', original_urls) | 20:37 |
| apollo13 | and a middleware which copies that path into a thread_local + a db backend | 20:37 |
| apollo13 | it might actually work ™ | 20:37 |
| * dmsimard looks up ROOT_URLCONF | 20:38 | |
| apollo13 | it's in the settings file and tells Django the entrypoint for routing | 20:38 |
| apollo13 | or we could hack around that in a wsgi app | 20:39 |
| apollo13 | so many options :D | 20:39 |
| apollo13 | I guess both approaches might work | 20:40 |
| apollo13 | I think rewriting the path and setting the threadlocal in a real wsgi app might be nicer | 20:41 |
| dmsimard | the thing with get_connection_params is that it's not in the request context at all | 20:42 |
| apollo13 | if we do it right we just have to rewrite script_name and path_info in the wsgi environment and django won't care | 20:42 |
| apollo13 | hence a thread local | 20:42 |
| dmsimard | that would be supplied by the middleware ? | 20:42 |
| apollo13 | let me show you some pseudo code | 20:42 |
| dmsimard | <3 | 20:43 |
| *** spiette has quit IRC | 20:43 | |
| *** spiette has joined #ara | 20:48 | |
| apollo13 | dmsimard: https://dpaste.de/5Jsx | 20:50 |
| apollo13 | oh we can even set https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-WSGI_APPLICATION to point to that and return django.core.wsgi.get_wsgi_application() instead of ara.server.wsgi.application | 20:52 |
| apollo13 | this way all you need then is a settings file which has WSGI_APPLICATION set to this alternative one and we could probably default the default sqlite backend to our backend that falls back to the path in settings if the thread local isn't set | 20:52 |
| apollo13 | + a setting for a "root path" for the sqlite stuff, because you probably don't want to start looking in / | 20:53 |
| apollo13 | (on the fs) | 20:53 |
| apollo13 | does that make any sense? | 20:53 |
| * dmsimard reads | 20:57 | |
| dmsimard | apollo13: ok, so in terms of plumbing | 20:59 |
| dmsimard | It means adding WSGI_APPLICATION to settings.py (which could be customized with settings.yaml?) -- set WSGI_APPLICATION to ara.server.other_wsgi which would be what you pasted | 21:03 |
| dmsimard | We need to have a custom database backend which retrieves the path of the database based on the local thread storage | 21:03 |
| apollo13 | yes, but we can use our custom database backend also instead of the normal sqlite backend | 21:04 |
| dmsimard | you'd default to this new backend ? | 21:04 |
| apollo13 | yes | 21:04 |
| apollo13 | if we write it clever enough it can just fall back to the configured path from settings if there is nothing in the thread_local | 21:05 |
| dmsimard | I suppose it would be one less step for users to setup | 21:05 |
| apollo13 | and we can even hide the actual WSGI_APPLICATION setting behind a boolean flag MASS_HOSTING? | 21:05 |
| apollo13 | and then WSGI_APPLICATION = 'ara.server.some_other_wsgi' if MASS_HOSTING else django.core.wsgi | 21:06 |
| apollo13 | we don't have to actually expose it I guess | 21:06 |
| dmsimard | I guess | 21:06 |
| apollo13 | or make MASS_HOSTING a string which points to the base path from where you wanna start searching sqlite files | 21:07 |
| apollo13 | I can't promise anything, but I'll give the wsgi stuff a try tomorrow | 21:07 |
| apollo13 | and now I am going to bed I guess, it isn't long to midnight anymore :D | 21:07 |
| dmsimard | apollo13: I'm probably going to take a stab at it, will add you to a review if I send something | 21:08 |
| apollo13 | cool! | 21:08 |
| apollo13 | & good night | 21:08 |
| dmsimard | thanks for the help, have nice dreams :p | 21:08 |
| *** etienne has quit IRC | 21:27 | |
| *** etienne has joined #ara | 21:28 | |
| *** harlowja has quit IRC | 22:47 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!