| *** daneyon has joined #openstack-keystone | 00:00 | |
| *** wchrisj has joined #openstack-keystone | 00:05 | |
| *** browne1 has quit IRC | 00:07 | |
| *** wchrisj has quit IRC | 00:09 | |
| *** browne has joined #openstack-keystone | 00:09 | |
| *** dstanek_zzz is now known as dstanek | 00:19 | |
| *** nkinder has joined #openstack-keystone | 00:30 | |
| ayoung | gyee, done. | 00:31 |
|---|---|---|
| *** daneyon_ has joined #openstack-keystone | 00:34 | |
| *** daneyon has quit IRC | 00:38 | |
| *** browne has quit IRC | 00:41 | |
| *** richm has quit IRC | 00:48 | |
| *** wchrisj has joined #openstack-keystone | 01:06 | |
| *** wchrisj has quit IRC | 01:10 | |
| *** diegows has quit IRC | 01:28 | |
| *** marcoemorais has quit IRC | 01:29 | |
| *** lbragstad has joined #openstack-keystone | 01:32 | |
| *** wchrisj has joined #openstack-keystone | 01:43 | |
| *** gokrokve_ has quit IRC | 01:45 | |
| *** gokrokve has joined #openstack-keystone | 01:47 | |
| *** praneshp has quit IRC | 01:47 | |
| *** wchrisj has quit IRC | 01:48 | |
| *** gokrokve has quit IRC | 01:51 | |
| *** wchrisj has joined #openstack-keystone | 01:54 | |
| *** wchrisj has quit IRC | 01:59 | |
| *** daneyon has joined #openstack-keystone | 02:04 | |
| *** daneyon has quit IRC | 02:05 | |
| *** daneyon_ has quit IRC | 02:06 | |
| *** gokrokve has joined #openstack-keystone | 02:16 | |
| *** gokrokve has quit IRC | 02:21 | |
| *** mberlin1 has joined #openstack-keystone | 02:27 | |
| *** mberlin has quit IRC | 02:28 | |
| ayoung | dolphm, morganfainberg, care to kick this one over the threshold https://review.openstack.org/#/c/80401/ ? bknudson wrote it, so it is not going to get any better reviewed than it is now | 02:35 |
| ayoung | dstanek, or you ^^ | 02:35 |
| lbragstad | ayoung: ++ that will be nice to have | 02:36 |
| ayoung | lbragstad, saw you +1ed it already | 02:36 |
| lbragstad | ayoung: I think I had one concern but it could be properly addressed in a separate patch | 02:37 |
| openstackgerrit | A change was merged to openstack/keystone: Code which gets elements of tree in ldap moved to a common method https://review.openstack.org/86302 | 02:37 |
| morganfainberg | ayoung, also +2/+A the universal newline stuff | 02:38 |
| morganfainberg | ayoung, so that should be gating now | 02:39 |
| ayoung | saw that. thanks | 02:39 |
| ayoung | jamielennox, what do you think about a temporary auth plugin for keystoneclient: negotiate-external. It will set up the negotiate call on the client side, and will set method = "external" | 02:40 |
| jamielennox | ayoung: can do, we still need requests-kerberos to get updated though right | 02:41 |
| jamielennox | what are you trying to do? | 02:41 |
| jamielennox | we don't have a CLI or auth_token loading from a plugin | 02:41 |
| *** stevemar has joined #openstack-keystone | 02:43 | |
| *** harlowja is now known as harlowja_away | 02:43 | |
| gyee | ayoung, thanks | 02:46 |
| openstackgerrit | A change was merged to openstack/python-keystoneclient: remove universal_newlines https://review.openstack.org/79411 | 02:46 |
| *** gyee has quit IRC | 02:47 | |
| ayoung | jamielennox, I was just thinking through. I probably will be writing a hack to Horizon to do S4U2Proxy | 02:47 |
| jamielennox | ayoung: you may as well use it for testing | 02:52 |
| jamielennox | i still think we should just do a 'kerberos' thing for longer term | 02:52 |
| jamielennox | given that we can't load it anyway | 02:52 |
| ayoung | jamielennox, I'll make the change, based on Jose's, and submit it. We can keep it as a WIP | 02:52 |
| jamielennox | also if you look at the from_conf stuff you should be able to see how to load it externally | 02:53 |
| *** praneshp has joined #openstack-keystone | 02:55 | |
| *** wchrisj has joined #openstack-keystone | 02:55 | |
| *** wchrisj has quit IRC | 02:57 | |
| *** wchrisj_ has joined #openstack-keystone | 02:57 | |
| *** praneshp_ has joined #openstack-keystone | 03:00 | |
| *** wchrisj_ has quit IRC | 03:01 | |
| *** praneshp has quit IRC | 03:01 | |
| *** praneshp_ is now known as praneshp | 03:01 | |
| *** nkinder has quit IRC | 03:02 | |
| *** openstackgerrit has quit IRC | 03:04 | |
| *** openstackgerrit has joined #openstack-keystone | 03:04 | |
| *** nkinder has joined #openstack-keystone | 03:08 | |
| openstackgerrit | ayoung proposed a change to openstack/python-keystoneclient: Compressed Signature and Validation https://review.openstack.org/71181 | 03:15 |
| *** gokrokve has joined #openstack-keystone | 03:19 | |
| *** david-lyle has joined #openstack-keystone | 03:19 | |
| *** gokrokve has quit IRC | 03:24 | |
| openstackgerrit | ayoung proposed a change to openstack/python-keystoneclient: Regions Management https://review.openstack.org/79096 | 03:30 |
| *** zhiyan_ is now known as zhiyan | 03:48 | |
| *** rwsu has quit IRC | 03:50 | |
| *** marcoemorais has joined #openstack-keystone | 03:52 | |
| *** david-lyle has quit IRC | 03:54 | |
| *** marcoemorais1 has joined #openstack-keystone | 03:56 | |
| *** marcoemorais has quit IRC | 03:57 | |
| *** topol has joined #openstack-keystone | 04:04 | |
| *** ilives has joined #openstack-keystone | 04:10 | |
| *** gokrokve has joined #openstack-keystone | 04:20 | |
| *** gokrokve has quit IRC | 04:24 | |
| *** ayoung is now known as ayoung_ZZZzz | 04:25 | |
| openstackgerrit | David Stanek proposed a change to openstack/keystone: Identity authentication now uses rotated passwords https://review.openstack.org/74447 | 04:43 |
| openstackgerrit | David Stanek proposed a change to openstack/keystone: Adds table and model for storing rotated passwords https://review.openstack.org/73368 | 04:43 |
| openstackgerrit | David Stanek proposed a change to openstack/keystone: Adds an extension to enable password rotation https://review.openstack.org/74623 | 04:43 |
| openstackgerrit | David Stanek proposed a change to openstack/keystone: Ignore broken endpoints in get_catalog https://review.openstack.org/81528 | 04:47 |
| openstackgerrit | David Stanek proposed a change to openstack/keystone: Ignore broken endpoints in get_v3_catalog https://review.openstack.org/81527 | 04:47 |
| *** andreaf has joined #openstack-keystone | 04:48 | |
| *** andreaf_ has joined #openstack-keystone | 04:49 | |
| *** andreaf has quit IRC | 04:53 | |
| openstackgerrit | David Stanek proposed a change to openstack/keystone: Identity authentication now uses rotated passwords https://review.openstack.org/74447 | 05:09 |
| openstackgerrit | David Stanek proposed a change to openstack/keystone: Adds an extension to enable password rotation https://review.openstack.org/74623 | 05:09 |
| *** bvandenh has joined #openstack-keystone | 05:11 | |
| *** shakamunyi has quit IRC | 05:14 | |
| *** gokrokve has joined #openstack-keystone | 05:20 | |
| *** gokrokve_ has joined #openstack-keystone | 05:22 | |
| *** gokrokve has quit IRC | 05:25 | |
| *** gokrokve_ has quit IRC | 05:27 | |
| *** topol_ has joined #openstack-keystone | 05:31 | |
| *** topol has quit IRC | 05:34 | |
| *** lbragstad has quit IRC | 05:36 | |
| *** topol_ has quit IRC | 05:37 | |
| openstackgerrit | A change was merged to openstack/keystone: Configurable token hash algorithm https://review.openstack.org/80401 | 05:42 |
| *** chandan_kumar has joined #openstack-keystone | 05:54 | |
| openstackgerrit | Sergey Nikitin proposed a change to openstack/keystone: Code which gets and deletes elements of tree was moved to one method https://review.openstack.org/86578 | 05:59 |
| *** bvandenh has quit IRC | 06:00 | |
| *** bvandenh has joined #openstack-keystone | 06:01 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/keystone: Imported Translations from Transifex https://review.openstack.org/88503 | 06:01 |
| *** ilives has quit IRC | 06:07 | |
| *** ilives has joined #openstack-keystone | 06:07 | |
| *** bvandenh has quit IRC | 06:15 | |
| *** bvandenh has joined #openstack-keystone | 06:15 | |
| *** gokrokve has joined #openstack-keystone | 06:23 | |
| *** gokrokve has quit IRC | 06:27 | |
| openstackgerrit | A change was merged to openstack/keystone: Include extra attributes in list results https://review.openstack.org/81041 | 06:35 |
| *** gokrokve has joined #openstack-keystone | 06:42 | |
| *** gokrokve has quit IRC | 06:42 | |
| *** gokrokve has joined #openstack-keystone | 06:42 | |
| *** gokrokve has quit IRC | 06:48 | |
| *** tomoiaga has joined #openstack-keystone | 06:54 | |
| openstackgerrit | Steve Martinelli proposed a change to openstack/python-keystoneclient: Add request/access token and consumer support for keystoneclient https://review.openstack.org/81980 | 07:13 |
| openstackgerrit | Steve Martinelli proposed a change to openstack/python-keystoneclient: Add request/access token and consumer support for keystoneclient https://review.openstack.org/81980 | 07:13 |
| *** dstanek is now known as dstanek_zzz | 07:16 | |
| *** stevemar has quit IRC | 07:20 | |
| *** ilives has quit IRC | 07:29 | |
| *** ilives has joined #openstack-keystone | 07:30 | |
| *** morganfainberg is now known as morganfainberg_Z | 07:38 | |
| *** ilives has quit IRC | 07:47 | |
| *** ilives has joined #openstack-keystone | 07:53 | |
| *** praneshp has quit IRC | 08:01 | |
| *** leseb has joined #openstack-keystone | 08:09 | |
| *** ilives has quit IRC | 08:20 | |
| *** ilives has joined #openstack-keystone | 08:22 | |
| tomoiaga | I see that v3/users/ {user_id} /roles is listed as a supported API call, however it seems to return a 404 when called. Looking at the code I can't find an implementation for it, maybe someone knows something about this. | 08:23 |
| *** marcoemorais1 has quit IRC | 08:35 | |
| *** ilives has quit IRC | 08:42 | |
| *** ilives has joined #openstack-keystone | 08:43 | |
| openstackgerrit | Matthieu Huin proposed a change to openstack/python-keystoneclient: Limited use trusts https://review.openstack.org/57492 | 08:43 |
| *** ilives has quit IRC | 08:50 | |
| *** ilives has joined #openstack-keystone | 08:52 | |
| *** bada has quit IRC | 09:00 | |
| *** marcoemorais has joined #openstack-keystone | 09:03 | |
| *** dstanek_zzz is now known as dstanek | 09:05 | |
| *** marcoemorais1 has joined #openstack-keystone | 09:05 | |
| *** marcoemorais has quit IRC | 09:08 | |
| *** marcoemorais1 has quit IRC | 09:09 | |
| *** Chicago has joined #openstack-keystone | 09:20 | |
| *** Chicago has joined #openstack-keystone | 09:20 | |
| *** dstanek is now known as dstanek_zzz | 09:23 | |
| *** ilives has quit IRC | 09:34 | |
| *** zhiyan is now known as zhiyan_ | 09:42 | |
| *** marcoemorais has joined #openstack-keystone | 10:06 | |
| *** marcoemorais has quit IRC | 10:10 | |
| *** fmarco76 has joined #openstack-keystone | 10:10 | |
| *** fmarco76 has left #openstack-keystone | 10:11 | |
| *** dstanek_zzz is now known as dstanek | 10:15 | |
| *** dstanek is now known as dstanek_zzz | 10:24 | |
| *** leseb has quit IRC | 10:33 | |
| *** leseb has joined #openstack-keystone | 10:34 | |
| *** leseb has quit IRC | 10:38 | |
| *** jaosorior has joined #openstack-keystone | 10:56 | |
| *** jamielennox is now known as jamielennox|away | 11:02 | |
| *** marcoemorais has joined #openstack-keystone | 11:07 | |
| *** leseb has joined #openstack-keystone | 11:09 | |
| *** marcoemorais has quit IRC | 11:11 | |
| *** leseb has quit IRC | 11:14 | |
| *** dstanek_zzz is now known as dstanek | 11:15 | |
| *** kashyap has joined #openstack-keystone | 11:15 | |
| jaosorior | Hey, I've noticed that the keystone.common.models is barely used anywhere except for the ldap backend | 11:21 |
| jaosorior | I was thinking of refactoring some modules in order to use more the information in the models | 11:21 |
| jaosorior | for example, keystone.common.identity.core contains a function called filter_user, where some fields in the User model are hardcoded, instead, I would like to get these attributes from the model itself. Is there any desire for something like this? | 11:23 |
| *** dstanek is now known as dstanek_zzz | 11:25 | |
| kashyap | Hi, while setting up Keystone endpoint, is v3.0 recommended or should I go w/ v2.0? This is w/ IceHouse | 11:31 |
| tomoiaga | kashyap: v3 is not yet supported by services like nova and the rest. You can setup both if you want to play around with v3 or give access to v3 to others but you will also need to have v2 for the other services | 11:32 |
| kashyap | tomoiaga, Okay, I'm just setting up a minimal IceHouse setup to do some bug triage, so I'll go w/ V2 | 11:33 |
| kashyap | Thank you for your response | 11:33 |
| tomoiaga | kashyap: np. I personally use both, but v2 needs to remains at least for services until Juno at least | 11:34 |
| kashyap | tomoiaga, Hmm, my minimal setup involves - Nova, Keystone, Neutron, Glance. Cinder is not really necessary in a minimal env, right? | 11:34 |
| tomoiaga | kashyap: no, unless you want to support volumes (centralized storage). You will be able to have local storage (ephemeral) without cinder so it should be ok. I have a setup working fine without cinder since I don't need volumes. | 11:36 |
| kashyap | tomoiaga, Yeah, I mostly use qcow2 files (preallocated), and nested virt (KVM-based). My entire setup is virtual. Thanks for this detail, again. | 11:37 |
| tomoiaga | kashyap: does it work ok for you in nested mode ? I never got the chance to test this but I would soon need to since I don't have unlimited hardware :) | 11:38 |
| * kashyap on the phone, be back in one minute | 11:38 | |
| tomoiaga | kashyap: I think I found your blog and I am going to go through your findings now :) | 11:39 |
| kashyap | tomoiaga, Yes, it does work for me :-) . I've been using nested virt consistently on Intel (some new hardware) ~ 2 years | 11:41 |
| kashyap | tomoiaga, I also have some newer notes here on a staging blog -- http://kashyap-testjekyll.rhcloud.com/blog/2014/01/30/nested-virt-with-virt-builderon-fedora/ | 11:42 |
| *** diegows has joined #openstack-keystone | 11:42 | |
| tomoiaga | kashyap: thank you! I'll read your blog and the newer notes since I need to catch up on some things. | 11:42 |
| kashyap | tomoiaga, Sure, no rush. Thanks to you too. | 11:43 |
| openstackgerrit | Li Ma proposed a change to openstack/keystone: Password trunction makes password insecure https://review.openstack.org/77325 | 11:53 |
| *** lbragstad has joined #openstack-keystone | 11:55 | |
| *** leseb has joined #openstack-keystone | 11:58 | |
| kashyap | tomoiaga, If you still have a moment, creating a Keystone service throws me -- http://paste.openstack.org/show/76773/ | 12:00 |
| kashyap | That's the command I invoked: $ keystone service-create --name keystone --type identity --description "Keystone Identity Service" | 12:01 |
| tomoiaga | kashyap: did you synced the db ? | 12:01 |
| tomoiaga | kashyap: I believe this is a new keystone installation, done manually right ? | 12:02 |
| kashyap | tomoiaga, That's what I ran -- keystone-manage db_sync keystone | 12:02 |
| kashyap | tomoiaga, Yes, done manually, I'm following my own notes from Havana - http://kashyapc.fedorapeople.org/virt/openstack/Two-node-Havana-setup.txt | 12:02 |
| tomoiaga | kashyap: can you run explain on the service column in your keystone db ? | 12:05 |
| kashyap | tomoiaga, Sorry, "run explain"? | 12:05 |
| *** dstanek_zzz is now known as dstanek | 12:05 | |
| tomoiaga | kashyap: login to mysql, use keystone (or the name of your db) and execute: explain service; | 12:05 |
| * kashyap tries | 12:06 | |
| tomoiaga | kashyap: explain will show the columns in the service table | 12:06 |
| kashyap | tomoiaga, I'm logged into mysql, do you have an exact command that I can try? (/me is db crippled) | 12:07 |
| tomoiaga | kashyap: you won't need openstack-neutron-openvswitch either on the controller node, unless you plan on adding instances there too. | 12:07 |
| *** marcoemorais has joined #openstack-keystone | 12:07 | |
| kashyap | tomoiaga, Good to know, will remove it from my IceHouse notes :-) | 12:08 |
| tomoiaga | kashyap: execute: use keystone (if keystone is the name of your db). After that execute: explain service | 12:08 |
| tomoiaga | kashyap: if you want to see all the databases, execute: show databases | 12:09 |
| kashyap | tomoiaga, Still trying, I'm on a remote machine, and I'm using a slow USB internet. . . | 12:09 |
| kashyap | MariaDB [keystone]> explain service; | 12:10 |
| kashyap | +-------+--------------+------+-----+---------+-------+ | 12:10 |
| kashyap | | Field | Type | Null | Key | Default | Extra | | 12:10 |
| kashyap | +-------+--------------+------+-----+---------+-------+ | 12:10 |
| kashyap | | id | varchar(64) | NO | PRI | NULL | | | 12:10 |
| kashyap | | type | varchar(255) | YES | | NULL | | | 12:10 |
| kashyap | | extra | mediumtext | YES | | NULL | | | 12:10 |
| kashyap | +-------+--------------+------+-----+---------+-------+ | 12:10 |
| kashyap | 3 rows in set (0.01 sec) | 12:10 |
| *** marcoemorais has quit IRC | 12:11 | |
| tomoiaga | kashyap: the enabled column seems to be missing. Try to run from ssh again and see if you notice any errors/warnings: keystone-manage db_sync | 12:12 |
| * kashyap re-tries: keystone-manage db_sync | 12:13 | |
| kashyap | tomoiaga, Yeah, it's now there: | 12:13 |
| kashyap | | enabled | tinyint(1) | NO | | 1 | | | 12:14 |
| kashyap | +---------+--------------+------+-----+---------+-------+ | 12:14 |
| tomoiaga | kashyap: good, db_sync needs to be executed when you first install keystone or after an upgrade | 12:14 |
| *** dstanek is now known as dstanek_zzz | 12:15 | |
| kashyap | tomoiaga, That seems weird -- why "sync" a db when you're installing for the first time | 12:16 |
| * kashyap now re-tries creating Keystone service | 12:17 | |
| tomoiaga | kashyap: well, the rpm package may not sync the db for you or it may, it depends on who configured the rpm, but in any case, if you have a remove mysql server, someone creating the rpm package can't know where that mysql server is to configure keystone and run the sync for you | 12:17 |
| kashyap | tomoiaga, I see, fair enough. Thanks for your time to help debug this. | 12:19 |
| tomoiaga | kashyap: glad it's working | 12:20 |
| openstackgerrit | Matthieu Huin proposed a change to openstack/keystone: More random values for oAuth1 verifier https://review.openstack.org/89612 | 12:22 |
| *** dstanek_zzz is now known as dstanek | 12:26 | |
| *** andreaf_ has quit IRC | 12:48 | |
| *** andreaf has joined #openstack-keystone | 12:48 | |
| *** marcoemorais has joined #openstack-keystone | 13:08 | |
| *** marcoemorais1 has joined #openstack-keystone | 13:10 | |
| *** dstanek is now known as dstanek_zzz | 13:11 | |
| *** marcoemorais has quit IRC | 13:12 | |
| *** dstanek_zzz is now known as dstanek | 13:14 | |
| *** marcoemorais1 has quit IRC | 13:14 | |
| *** daneyon has joined #openstack-keystone | 13:19 | |
| *** daneyon has quit IRC | 13:22 | |
| *** daneyon has joined #openstack-keystone | 13:23 | |
| *** jzl_ has joined #openstack-keystone | 13:24 | |
| *** jzl_ has quit IRC | 13:24 | |
| *** afaranha has quit IRC | 13:34 | |
| *** thiagop has quit IRC | 13:34 | |
| *** afaranha has joined #openstack-keystone | 13:40 | |
| *** joesavak has joined #openstack-keystone | 13:42 | |
| *** thiagop has joined #openstack-keystone | 13:43 | |
| *** chandan_kumar is now known as chandankumar | 13:54 | |
| *** wchrisj has joined #openstack-keystone | 13:56 | |
| *** stevemar has joined #openstack-keystone | 13:57 | |
| *** gokrokve has joined #openstack-keystone | 14:02 | |
| openstackgerrit | Steve Martinelli proposed a change to openstack/python-keystoneclient: Add request/access token and consumer support for keystoneclient https://review.openstack.org/81980 | 14:03 |
| *** topol has joined #openstack-keystone | 14:04 | |
| *** marcoemorais has joined #openstack-keystone | 14:11 | |
| openstackgerrit | ayoung proposed a change to openstack/keystone: Remember the DN https://review.openstack.org/47441 | 14:14 |
| *** rwsu has joined #openstack-keystone | 14:14 | |
| *** marcoemorais has quit IRC | 14:15 | |
| *** thedodd has joined #openstack-keystone | 14:22 | |
| *** bada has joined #openstack-keystone | 14:48 | |
| openstackgerrit | Steve Martinelli proposed a change to openstack/keystone: Add detailed federation configuration docs https://review.openstack.org/89220 | 14:54 |
| *** david-lyle has joined #openstack-keystone | 14:55 | |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Sync test_migrations https://review.openstack.org/80618 | 15:12 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Redundant unique constraint https://review.openstack.org/84447 | 15:12 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Corresponding `nullable` value. https://review.openstack.org/84446 | 15:12 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Migration DB_INIT_VERSION in common place https://review.openstack.org/88016 | 15:12 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Compatible server default value in the models. https://review.openstack.org/84445 | 15:12 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Explicit foreign key indexes. https://review.openstack.org/84444 | 15:12 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Sync on-demand database schemas https://review.openstack.org/84448 | 15:12 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations. https://review.openstack.org/80630 | 15:12 |
| *** dstanek is now known as dstanek_zzz | 15:17 | |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Sync test_migrations https://review.openstack.org/80618 | 15:17 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Redundant unique constraint https://review.openstack.org/84447 | 15:17 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Corresponding `nullable` value. https://review.openstack.org/84446 | 15:17 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Migration DB_INIT_VERSION in common place https://review.openstack.org/88016 | 15:17 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Compatible server default value in the models. https://review.openstack.org/84445 | 15:17 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Explicit foreign key indexes. https://review.openstack.org/84444 | 15:17 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Sync on-demand database schemas https://review.openstack.org/84448 | 15:17 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations. https://review.openstack.org/80630 | 15:17 |
| *** gokrokve_ has joined #openstack-keystone | 15:31 | |
| *** gokrokve has quit IRC | 15:34 | |
| *** browne has joined #openstack-keystone | 15:34 | |
| openstackgerrit | Steve Martinelli proposed a change to openstack/keystone: Rename extension docs from configure to enable https://review.openstack.org/89882 | 15:37 |
| *** gyee has joined #openstack-keystone | 15:40 | |
| stevemar | marekd|away, thanks for the comments :) | 15:43 |
| *** jaosorior has quit IRC | 16:01 | |
| *** marekd|away is now known as marekd | 16:02 | |
| marekd | stevemar: i wrote them in the plane, just got connected and uploaded without actually looking that you uploaded another patchset. | 16:03 |
| stevemar | marekd, i'll forgive you, being on a plane and all | 16:03 |
| marekd | uffff | 16:03 |
| *** marcoemorais has joined #openstack-keystone | 16:04 | |
| marekd | stevemar: i am not sure if we can say that there is a possibility to map federated users to existing users... | 16:11 |
| marekd | stevemar: i am assuming this would be happening just because the user ids match, but do you think it's enough? | 16:11 |
| *** jsavak has joined #openstack-keystone | 16:12 | |
| marekd | stevemar: AFAIR the whole federation thing was made and we avoided user lookups in the backend... | 16:13 |
| *** joesavak has quit IRC | 16:13 | |
| marekd | user_id produced by the maping engine is just an information, something that may help in accounting or something like that. | 16:14 |
| *** gokrokve has joined #openstack-keystone | 16:14 | |
| *** joesavak has joined #openstack-keystone | 16:15 | |
| *** richm has joined #openstack-keystone | 16:15 | |
| *** gokrokv__ has joined #openstack-keystone | 16:16 | |
| stevemar | marekd, i agree, I thought I wrote that back in my comment. It's just for auditing | 16:17 |
| *** Chicago has quit IRC | 16:18 | |
| *** jsavak has quit IRC | 16:18 | |
| *** gokrokve_ has quit IRC | 16:18 | |
| marekd | stevemar: ok, that 'sort of' was confusing :P | 16:18 |
| stevemar | marekd, ah, that was a nice way for me to say 'no' | 16:20 |
| *** gokrokve has quit IRC | 16:20 | |
| marekd | ok ok :-) | 16:21 |
| *** tomoiaga has quit IRC | 16:22 | |
| *** Anju_ has joined #openstack-keystone | 16:22 | |
| *** marekd is now known as marekd|afk | 16:27 | |
| *** thiagop has quit IRC | 16:28 | |
| *** erecio has joined #openstack-keystone | 16:33 | |
| *** kashyap has left #openstack-keystone | 16:33 | |
| *** leseb has quit IRC | 16:40 | |
| *** amcrn has joined #openstack-keystone | 16:47 | |
| openstackgerrit | Doug Hellmann proposed a change to openstack/keystone: Register all backend classes as entry points https://review.openstack.org/89419 | 16:52 |
| *** chandankumar has quit IRC | 16:53 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/keystone: Updated from global requirements https://review.openstack.org/89235 | 16:54 |
| *** ayoung_ZZZzz is now known as ayoung | 16:55 | |
| ayoung | dhellmann, when I hear Entrypoints I think exported symbols, and I never quite got why. Why? | 16:59 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/python-keystoneclient: Updated from global requirements https://review.openstack.org/89245 | 17:01 |
| *** harlowja_away is now known as harlowja | 17:06 | |
| *** spligak has joined #openstack-keystone | 17:09 | |
| *** gokrokv__ has quit IRC | 17:15 | |
| openstackgerrit | Steve Martinelli proposed a change to openstack/python-keystoneclient: Add request/access token and consumer support for keystoneclient https://review.openstack.org/81980 | 17:16 |
| *** gokrokve has joined #openstack-keystone | 17:17 | |
| *** dstanek_zzz is now known as dstanek | 17:20 | |
| *** gokrokve has quit IRC | 17:23 | |
| lbragstad | ayoung: had a couple very minor/trivial comments on https://review.openstack.org/#/c/81166/ | 17:24 |
| lbragstad | I can push a new patch if you want | 17:24 |
| ayoung | lbragstad, must be something with my editor. I copied and pasted the "good" one over the overly spaced one. | 17:25 |
| *** praneshp has joined #openstack-keystone | 17:25 | |
| ayoung | Nah, I'll repush... | 17:25 |
| lbragstad | ayoung: cool, what are you using for editor? just curious | 17:26 |
| ayoung | PyCharm | 17:26 |
| dstanek | ayoung: how do you like PyCharm? | 17:26 |
| lbragstad | ah gotcha | 17:26 |
| ayoung | dstanek, I like it for the most part. Good browsing. Its a bit of a pain wokring with multiple projects, as each needs to be open in its own window, but that is almost more a feature. Debugging support is good | 17:29 |
| ayoung | automated refactorings are better than in Eclipse pydev. | 17:30 |
| openstackgerrit | guang-yee proposed a change to openstack/keystone: Make sure all the auth plugins agree on the shared identity attributes. https://review.openstack.org/84945 | 17:32 |
| ayoung | nkinder, I'm getting lost looking at the rebase of "Remember the DN" llist_group_users https://review.openstack.org/#/c/47441/12/keystone/identity/backends/ldap.py | 17:34 |
| ayoung | lbragstad, the copyright headers are formatted the same as other files in the same subdir | 17:36 |
| ayoung | I guess it is a question of which spacing is right... | 17:37 |
| lbragstad | ayoung: yeah I'm guess I'm not sure which spacing is 'right' | 17:37 |
| ayoung | http://docs.openstack.org/developer/hacking/#openstack-licensing lbragstad | 17:37 |
| *** chandan_kumar has joined #openstack-keystone | 17:39 | |
| lbragstad | so two spaces before text | 17:40 |
| nkinder | ayoung: checking... | 17:42 |
| nkinder | ayoung: ok, where are you getting lost? | 17:43 |
| ayoung | nkinder, so the code that messed up the rebase is new | 17:43 |
| ayoung | and I don;t think it was there before you did your work. So which approach do we follow | 17:43 |
| ayoung | group_dn = self._id_to_dn(group_id) is what we want to repalce, right? | 17:44 |
| ayoung | replace | 17:44 |
| ayoung | I dropped the "query" param on the rebase, which I think was just a mistake in merging | 17:44 |
| nkinder | ayoung: which code is new? The use of _ldap_get_list()? | 17:45 |
| ayoung | nkinder, I think so, yeah | 17:45 |
| nkinder | ayoung: so here's how it was when I made my patch - https://review.openstack.org/#/c/47441/6/keystone/identity/backends/ldap.py | 17:46 |
| nkinder | ayoung: I was avoiding _id_to_dn() | 17:46 |
| ayoung | nkinder, so keep the existing code for the most part, but | 17:46 |
| ayoung | use group_ref = self.get(group_id) | 17:46 |
| ayoung | 388 group_dn = group_ref['dn'] | 17:47 |
| *** morganfainberg_Z is now known as morganfainberg | 17:48 | |
| nkinder | ayoung: keep the use of _ldap_get_list(), but extract the group_dn as I did in my patch | 17:49 |
| ayoung | ok...new patch in a moment | 17:49 |
| morganfainberg | bknudson, ping http://dal05.objectstorage.softlayer.net/v1/AUTH_58396f85-2c60-47b9-aaf8-e03bc24a1a6f/cilog/16/88016/6/check/ibm-db2-ci/1d58b38/console.html looks like there is possibly an error win a setup script w/ the db2 ci. | 17:51 |
| morganfainberg | bknudson, looking at some of the db changes and want to make sure db2 ci is running sanely (or know if we should ignore it in this case) | 17:51 |
| bknudson | /opt/stack/new/devstack-gate/devstack-vm-gate.sh: line 123: [: : integer expression expected | 17:52 |
| morganfainberg | bknudson, yep | 17:52 |
| morganfainberg | bknudson, not sure if that is something we (read: openstack-infra) did or something the team over there needs to look at | 17:52 |
| bknudson | morganfainberg: our team should look at it ... unless all the jobs are failing | 17:53 |
| morganfainberg | bknudson, let me check. | 17:53 |
| bknudson | Saving to: `/tmp/cacert.pem' -- that looks safe! | 17:53 |
| nkinder | ayoung: http://fpaste.org/96333/75662139/ | 17:54 |
| morganfainberg | bknudson, yeah wow, super safe! | 17:55 |
| bknudson | it's the public key, so not the worst thing | 17:55 |
| bknudson | just don't write a test that dumps that file! | 17:55 |
| ayoung | nkinder, yep, thjat is what I have, just testing | 17:55 |
| morganfainberg | bknudson, yeah looks like db2 ci is the one failing, upstream gate doesn't look to be having errors | 17:57 |
| openstackgerrit | ayoung proposed a change to openstack/keystone: Remember the DN https://review.openstack.org/47441 | 18:03 |
| morganfainberg | bknudson, i'll look at the code and give a +1 with a +2 note if db2 runs clean (either manually ot the ci) before merge. | 18:04 |
| bknudson | morganfainberg: which was the review? | 18:05 |
| bknudson | I'll make sure they do a recheck | 18:05 |
| morganfainberg | bknudson, there are a bunch by ilya pekelny that are around the DB. | 18:05 |
| bknudson | ok | 18:06 |
| morganfainberg | bknudson, i'll give you a list as i review them if it helps. | 18:06 |
| bknudson | I know which they are. | 18:06 |
| ayoung | nkinder, ^^ look right? | 18:06 |
| bknudson | I don't live under a rock. | 18:06 |
| morganfainberg | bknudson, hehe but rocks are cool | 18:08 |
| *** gokrokve has joined #openstack-keystone | 18:16 | |
| *** gokrokve has quit IRC | 18:21 | |
| morganfainberg | bknudson, somedays i think living under a rock might be better >.> | 18:21 |
| dstanek | morganfainberg: technology is just a dead end | 18:24 |
| morganfainberg | dstanek, i KNEW IT! | 18:24 |
| nkinder | ayoung: what's the point of calling self.get_group(group_id) in add_user_to_group()? | 18:25 |
| * morganfainberg invests heavily in rocks. | 18:25 | |
| nkinder | ayoung: that call was already there, but is it really needed? | 18:25 |
| ayoung | nkinder, no idea. You wrote that code, and I was just trying to rebase. | 18:26 |
| ayoung | Ah | 18:26 |
| nkinder | no, I didn't add it IIRC | 18:26 |
| ayoung | I don't think I did either, but let me look | 18:26 |
| dstanek | nkinder: a lot of times we do a get_* call and don't do anything with the value to make sure it actually exists | 18:27 |
| ayoung | nkinder, its probably checking for existance. | 18:27 |
| morganfainberg | nkinder, that is to raise an exception to check existence | 18:27 |
| nkinder | won't we find out when the MOD operation fails? | 18:27 |
| morganfainberg | nkinder, erm, s/raise exception// | 18:27 |
| nkinder | that will get an LDAP err=32 (LDAP_NO_SUCH_OBJECT), which should trigger an exception | 18:27 |
| ayoung | nkinder, might be the wrong exception | 18:28 |
| nkinder | all this does is cause an extra search | 18:28 |
| bknudson | write the tests and make sure they pass. | 18:28 |
| ayoung | yeah, we can probably streamline it | 18:28 |
| morganfainberg | ayoung, ++ probably GROUP_NOT_FOUND vs LDAP_ERROR | 18:28 |
| bknudson | have a test that the op fails if the entry doesn't exist | 18:28 |
| nkinder | less operations == profit | 18:28 |
| ayoung | bknudson, we need LDAP in the gate | 18:28 |
| ayoung | live LDAP | 18:28 |
| nkinder | ayoung: I'll test it out | 18:28 |
| bknudson | I think this one is handled by fakeldap. it's a pretty easy one | 18:29 |
| bknudson | it shouldn't be hard to write a fakeldap that fails if the entry doesn't exist, anyways | 18:29 |
| morganfainberg | nkinder, this also might just be an artifact of SQLisms where we do need to check for existence | 18:29 |
| bknudson | although it's pretty much impossible to compare DNs correctly | 18:29 |
| morganfainberg | nkinder, because the way RDBMs works/stores data vs. LDAP (you could create the row in sQL...) | 18:30 |
| nkinder | I want to see what this patch does with wireshark anyway, so I'll test without those calls with a live LDAP server and see what happens | 18:31 |
| morganfainberg | bknudson, is the DN comparison pain just python-isms? or a universal problem? | 18:31 |
| bknudson | morganfainberg: it's LDAP and not python-isms. | 18:32 |
| bknudson | 1) attribute names can take several forms (short names, long names, OIDs) | 18:32 |
| morganfainberg | bknudson, i've only worked with LDAP in C/C++ and Python. | 18:32 |
| morganfainberg | bknudson, i know it was a pain in both, but it might have been the libs we used in C/C++ | 18:33 |
| bknudson | 2) attribute values have different comparison rules (case-sensitive, case-insensitive, whatever) | 18:33 |
| morganfainberg | bknudson, ah yeah | 18:33 |
| bknudson | so you need the schema and also lots of code to be able to compare DNs | 18:33 |
| bknudson | and I don't think LDAP has a standard even for how to fetch the schema or what the schema looks like | 18:33 |
| morganfainberg | bknudson, yeah sounds about right | 18:33 |
| bknudson | something you'd think would be simple and fundamental to working with LDAP is pretty much impossible | 18:34 |
| morganfainberg | bknudson, sure it does... if you're an LDAP server and already know how to parse a schema... :P | 18:34 |
| morganfainberg | and using an opensource LDAP server with it's own protocol for that. | 18:34 |
| bknudson | maybe ldap has an extended op for comparing dns | 18:35 |
| bknudson | would expect that to be slow since it requires a roundtrip to the server, though | 18:35 |
| nkinder | ayoung: I found one error in the patch. I'll run some tests with it now and will submit a new rev. afterwards | 18:35 |
| morganfainberg | bknudson, hm. | 18:35 |
| nkinder | bknudson: there is a compare operation | 18:36 |
| nkinder | it's not commonly used though | 18:36 |
| bknudson | attribute compare, e.g., for the password | 18:36 |
| morganfainberg | bknudson, i always found the python ldap impl (python-ldap lib) more kludgy than some other options | 18:36 |
| morganfainberg | nkinder, it sounds expensive... lets ask the LDAP server to compare these things. | 18:36 |
| morganfainberg | nkinder, oh another OP every time. | 18:36 |
| nkinder | yeah, definitely | 18:36 |
| nkinder | DNs can be easily compared by doing a base level search | 18:37 |
| nkinder | still a round-trip | 18:37 |
| morganfainberg | nkinder, hm. still yeah | 18:37 |
| bknudson | and the entry has to exist. | 18:37 |
| nkinder | If you pass a legal DN, an LDAP server is supposed to normalize it and do a syntax-aware comparison when searching | 18:37 |
| nkinder | bknudson: yes, absolutely | 18:38 |
| bknudson | let's say I want to find out if this entry would be in this subtree, for example. | 18:38 |
| bknudson | (although that might not even make a lot of sense if you've got aliases and references) | 18:38 |
| nkinder | the client would be responsible to be smart about the DN syntax in that case | 18:39 |
| nkinder | which is easier said than done | 18:39 |
| bknudson | luckily DNs tend to only use a few attrs, so if you can limit yourself to that then you can make something that works client-side | 18:39 |
| *** chandan_kumar has quit IRC | 18:41 | |
| *** gokrokve has joined #openstack-keystone | 18:45 | |
| ayoung | https://launchpad.net/~ayoung I likes my new Icon! | 18:47 |
| morganfainberg | ayoung, nice | 18:47 |
| ayoung | morganfainberg, I love howlaunchpad calls it "rebranding" | 18:48 |
| morganfainberg | wait... they call it rebranding? | 18:48 |
| * morganfainberg snickers | 18:48 | |
| nkinder | do I remember correctly that a change was made a little while back that eliminates the need to mount keystone/tests/tmp as tmpfs to speed up the tests? | 18:52 |
| morganfainberg | nkinder, correct. | 18:52 |
| nkinder | yay, I'm not 100% crazy | 18:52 |
| morganfainberg | nkinder, we do all the SQL-lite migrations/db work in-memory now | 18:52 |
| morganfainberg | the #1 reason to mnt tmpfs was for the db file | 18:53 |
| morganfainberg | there are some files that are still dropped on disk, but they are minor and the tiny improvements (unless you're really io-bound) shouldn't be noticable | 18:53 |
| nkinder | does this test failure look familiar to anyone? http://paste.openstack.org/show/76825/ | 18:58 |
| nkinder | I have a freshly installed OS, so it may be environment related. That said, I was able to run all of the unit tests in stable/icehouse earler this morning on this system. | 18:58 |
| morganfainberg | nkinder, looking now | 19:00 |
| dstanek | nkinder: that's pretty odd | 19:00 |
| morganfainberg | nkinder, yeah i know what that issue is | 19:00 |
| dstanek | morganfainberg: ? | 19:01 |
| morganfainberg | dstanek, he didn't use 'usedevelop' to install keystone | 19:01 |
| nkinder | ? | 19:01 |
| morganfainberg | it's the way we load the files on the backend | 19:01 |
| morganfainberg | it's a bug | 19:01 |
| morganfainberg | https://review.openstack.org/#/c/89419/ | 19:01 |
| morganfainberg | that patch fixes it | 19:01 |
| morganfainberg | or should | 19:01 |
| morganfainberg | basically it is looking in the wrong place for the code to do reflection based schema building | 19:02 |
| morganfainberg | the code assumes you're using the pip -e . style setup (or similar) for running tests | 19:02 |
| morganfainberg | there are other ways it can conflict, the answer is to use stevedore and entry points instead | 19:02 |
| morganfainberg | makes things a lot less likely to break in odd ways | 19:02 |
| dstanek | morganfainberg: really? it should be looking relative to the tests | 19:03 |
| morganfainberg | dstanek, dhellmann ran across the same issue | 19:03 |
| nkinder | for setup, I created a venv, then just did 'pip -r' for requirements.txt and test-requirements.txt | 19:03 |
| nkinder | after that, I'm just using run_tests.sh | 19:04 |
| morganfainberg | nkinder, if you did pip install -e . | 19:04 |
| morganfainberg | that would then fix it. | 19:04 |
| morganfainberg | i think that is the issue. | 19:04 |
| nkinder | morganfainberg: ++ | 19:04 |
| nkinder | morganfainberg: that step won't be necessary once the above patch is merged? | 19:05 |
| dstanek | i'll have to try to recreate - i use tox and have not actually installed keystone into a venv | 19:05 |
| morganfainberg | nkinder, hopefully | 19:05 |
| morganfainberg | nkinder, though to be fair, that is how gate does it, (run_tests should do the same) | 19:06 |
| morganfainberg | nkinder, iirc tox also works like that. | 19:06 |
| dstanek | morganfainberg: how does that patch know to import from the backends package? | 19:07 |
| dstanek | the thing i don't like about that patch is i think it'll need a test to make sure as new sql modules are added then get added to the list | 19:07 |
| morganfainberg | dstanek, it's based on the namespace list and the use of entrypoints | 19:08 |
| morganfainberg | dstanek, the entrypoint says "keystone.assignment" and setup.cfg has the mapping to keystone.assignment.backends for each module | 19:08 |
| dstanek | morganfainberg: ah i didn't look in the setup.cfg | 19:08 |
| *** bvandenh has quit IRC | 19:09 | |
| morganfainberg | dstanek, we might need a test to make sure things end up in that lilst, but there was a developing.rst change as well, and the schema wont exist (you'll get other failures for SQL tests) | 19:09 |
| morganfainberg | dstanek, as in SQLA - no such table | 19:09 |
| dstanek | morganfainberg: that's the problem - you won't :-( | 19:09 |
| dstanek | a test will likely import the sql before it's used | 19:09 |
| morganfainberg | dstanek, the test absolutely would fail if you don't import the module | 19:09 |
| dstanek | but that mean the schema would be different between the tests and depend on the ordre | 19:10 |
| morganfainberg | hmmm | 19:10 |
| dstanek | the point of what i did was to make sure the schema is always the same | 19:10 |
| bknudson | dstanek: could you write a check to ensure that no new models were imported by the test? | 19:12 |
| dstanek | bknudson: not really because by the time the test runs they have already been imported | 19:12 |
| bknudson | check before and after | 19:12 |
| morganfainberg | there has to be abetter way than walking the directory. | 19:12 |
| morganfainberg | or stevedore | 19:12 |
| morganfainberg | i think i know why nova sticks all it's models into a single module now. | 19:13 |
| dstanek | technically you could check and the begining of each test and the first test could have off the list | 19:13 |
| dstanek | bknudson: the sql modules will be import before the test runs, not during the test run | 19:14 |
| morganfainberg | maybe the right answer is to have a single location all models go. | 19:14 |
| dstanek | bknudson: well, not all of the time; but generally speaking | 19:14 |
| morganfainberg | move them to like keystone/common/sql/models/<module_name>.py ? | 19:14 |
| bknudson | morganfainberg: what about extensions? | 19:14 |
| dstanek | morganfainberg: what about extensions? | 19:14 |
| bknudson | jinx! | 19:14 |
| bknudson | quit reading my mind | 19:15 |
| morganfainberg | bknudson, dstanek, for in-tree extensions we have prior art for that - the doc tree | 19:15 |
| morganfainberg | bknudson, dstanek, and i was actually typing the response to that before you guys asked :P | 19:15 |
| morganfainberg | not sure i like that structure, but it could be worse. | 19:16 |
| dstanek | i have to think about that a bit | 19:16 |
| dstanek | i'm just not sure about stevedor based on the way we wire extensions via paste | 19:17 |
| bknudson | since we test with all the models we should probably run with all the models | 19:17 |
| morganfainberg | bknudson, that is kindof my thought | 19:17 |
| bknudson | and we test with all the tables created so should probably run that way | 19:17 |
| bknudson | or fix the tests so that it works the way the server runs | 19:17 |
| bknudson | this is why global state is a disaster waiting to happen | 19:18 |
| *** leseb has joined #openstack-keystone | 19:18 | |
| morganfainberg | bknudson, the way SQLA registers models / table defs | 19:20 |
| morganfainberg | bknudson, i think the answer is to run with 100% of the models and the schema in the db | 19:20 |
| dstanek | morganfainberg: so for real deployments have all tables for all extensions loaded? | 19:21 |
| morganfainberg | bknudson, it doesn't hurt anything to have the extra tables for tests (we don't check that tables exist/dont) | 19:21 |
| morganfainberg | dstanek, that is a migrate question | 19:21 |
| morganfainberg | dstanek, real deployments use migrate. | 19:21 |
| morganfainberg | dstanek, we could go back to migrating everytime. | 19:22 |
| ayoung | morganfainberg, is stevedore the path forward? dstanek it seems to be in the same general field as your snake-guice approach | 19:22 |
| dstanek | what problem are we trying to solve now? seems like we got away from the directory crawling | 19:23 |
| morganfainberg | dstanek, it probably wouldn't be bad to just create the tables for all extensions everytime anyway - it doesn't cost a lot and it is less likely ot cause issues with DBs having a new extension enabled and not having the schema to support it | 19:23 |
| ayoung | Pub crawling! | 19:23 |
| ayoung | oops, sorry | 19:23 |
| *** bach has joined #openstack-keystone | 19:23 | |
| morganfainberg | ayoung, stevedore for loading the backend is the "openstack" way (vs. importutils?) | 19:23 |
| dstanek | ayoung: not really; it's about extensions not about IoC | 19:23 |
| ayoung | dstanek, not as I read it | 19:23 |
| morganfainberg | ayoung, it just happens to work for this usecase (sortof) | 19:23 |
| ayoung | dstanek, they are two sides of the same coin | 19:23 |
| ayoung | dstanek, ok, lets start with entrypoints. What are they good for? | 19:24 |
| dstanek | ayoung: i don't think they are - it doesn't construct object graphs | 19:24 |
| morganfainberg | ayoung, entrypoints solve a different issue, simplified namespace + objects (and/or scripts) | 19:24 |
| morganfainberg | ayoung, it also could allow other modules (3rd party) to supply backends to keystone by using the correct namespace | 19:25 |
| morganfainberg | ayoung, e.g. keystone.assignment backendModule My_Cool_assignment_backend | 19:25 |
| dstanek | morganfainberg: the thing with that is that we currently put the fully qualified module path in the config | 19:25 |
| morganfainberg | and you just ask for that vs needing to know <where>.<did>.<i>.<install>.my_cool_backend | 19:26 |
| ayoung | the thing is, I think entrypoints are just another way of naming dependencies | 19:26 |
| morganfainberg | ayoung, uhm could be? | 19:26 |
| ayoung | so " I need a database connection" vs "I am a database connection" | 19:26 |
| dstanek | ayoung: it's much higher level | 19:26 |
| ayoung | entrypoint=databaseconnection | 19:26 |
| morganfainberg | ayoung, sure you could do that | 19:26 |
| morganfainberg | but i think dstanek is right | 19:26 |
| bknudson | morganfainberg: I just did a recheck db2-test and it worked this time. | 19:27 |
| dstanek | you wouldn't create an entrypoint for every interface | 19:27 |
| bknudson | must have healed itself | 19:27 |
| morganfainberg | bknudson, cool. ok no concerns then. | 19:27 |
| dstanek | and you wouldn't want more than one thing registered for an entrypoint | 19:27 |
| morganfainberg | bknudson, thanks :) (hadn't gotten to review yet, was on other discussions first) | 19:27 |
| ayoung | morganfainberg, dstanek I don't want two mechanisms where one will do, and I want the one that is the most "core" | 19:28 |
| dstanek | ayoung: a quick example is ICatalogBackend (i hate the I naming convention, but it's easier to visualize) | 19:28 |
| ayoung | From what I've read of entrypoints, its for componentn naming | 19:28 |
| dstanek | which one do you pick? all 3 are registered under stevedore (and maybe thirdpary ones too) | 19:28 |
| morganfainberg | ayoung, you need to be specific about which one tyou're loading, the entry point just saves needing to know the full path for every possible variation | 19:29 |
| ayoung | dstanek, you, how do you distinguish between three components that all fulfill the same interface contract? | 19:29 |
| dstanek | ayoung: exactly. we do that in the config file | 19:29 |
| bknudson | so would we have something like [assignment] driver=sql ? | 19:29 |
| morganfainberg | ayoung, you ask for <namespace [ like keystone.assignment ]> with the module SQL(or KVS) | 19:29 |
| morganfainberg | bknudson, yep. and we'd look for keystone.assignment namespace, module sql | 19:30 |
| dstanek | entrypoints are really good for dynamic discovery questions like 'show me a list of all identity backends' | 19:30 |
| morganfainberg | bknudson, the namespace would be known and we always look in that namespace for an assignment module | 19:30 |
| dstanek | not so good at instantiating classes | 19:30 |
| ayoung | I'm just learning stevedore, and I have not yes groked it, nor do I have a position on how well or poorly it is designed, but since it looks like it is already integrated its way into opnestack, I'm trying to understand it. Stevedore seems to make heavy use of entrypoints, and so I m trying to get a good sense of them. | 19:30 |
| morganfainberg | so rax could have keystone.assignment <RAX> | 19:30 |
| bknudson | and [token] provider=uuid | 19:30 |
| bknudson | that would be nice. | 19:30 |
| morganfainberg | bknudson, ++ yep | 19:31 |
| ayoung | From what little I've read, there is a concept of namespaces. I would say that you use namespaces as one way to distinguish | 19:31 |
| morganfainberg | bknudson, that solves a separate problemspace from i think the test one | 19:31 |
| dstanek | bknudson: that's how we would probably use it | 19:31 |
| ayoung | Its not an easy problem. At its most complex you have a query language, and you use that to select the component you want | 19:31 |
| morganfainberg | would it hurt us to always migrate all extensions? and run with all models? | 19:31 |
| ayoung | based on "criteria" | 19:32 |
| dstanek | then code would look to see what was registered as sql for the tokenbackend | 19:32 |
| bknudson | so we're already using entry points, for the cli | 19:32 |
| morganfainberg | back to the origional discussion | 19:32 |
| bknudson | is that stevedore already? | 19:32 |
| ayoung | alternatively, you annotate a component. For example, with databse conections, in an ecommerce site you can have "catalog, orders, browsing" | 19:32 |
| morganfainberg | bknudson, i don't think so w/ keystoneclient | 19:32 |
| ayoung | each are tuned differently | 19:33 |
| dstanek | bknudson: no that setuptools | 19:33 |
| morganfainberg | bknudson, but we might be? | 19:33 |
| ayoung | so a given component that needs both catalog and orders should be able to say "I need two different data base connections. One is the order one..." | 19:33 |
| dstanek | ayoung: that is what you would do in an IoC framework | 19:34 |
| ayoung | dstanek, RIGHT | 19:34 |
| ayoung | dstanek, but the component naming standard seems to be "entrypoints" | 19:34 |
| morganfainberg | ayoung, i think entrypoints are more for providing options in a given namespace but not providing _the_ option. | 19:34 |
| dstanek | morganfainberg: exactly | 19:34 |
| dstanek | you still need something to say "i use the sql catalog" | 19:35 |
| morganfainberg | ayoung, you're looking for a single option here, stevedore allows you to register something clearly into a namespace and make it available without needing to know it.is.at.this.path.object | 19:35 |
| morganfainberg | dstanek, ++ | 19:35 |
| morganfainberg | ayoung, exactly what dstanek just said | 19:35 |
| ayoung | morganfainberg, yes. Which to me sounds like "neither necessary nor sufficient" | 19:35 |
| ayoung | so why use it? | 19:35 |
| morganfainberg | we can table that discussion and bring in common code folks (dhelmann) on why/why-not use that | 19:36 |
| dstanek | we can use entrypoints without stevedor; including what dhellmann's patch does | 19:37 |
| dstanek | i have to look into stevedor to see what it actually does on top on entrypoints | 19:37 |
| morganfainberg | i think we need to (more importantly) solve the model issue wetherornot it's entrypoints / stevedore | 19:37 |
| ayoung | What is the "model" issue? | 19:37 |
| morganfainberg | ayoung, we need to import all models (SQLA) for testing so db schema matches with reflection | 19:38 |
| morganfainberg | ayoung, erm so db schema matches in all tests because we use reflection to create it | 19:39 |
| ayoung | cuz I have to say, I really don't like the look of that setup.cfg | 19:39 |
| morganfainberg | excluding the migration_specific tests | 19:39 |
| morganfainberg | otherwise the db schema could vary on any given test. | 19:39 |
| morganfainberg | if a model has been imported or not | 19:39 |
| dstanek | morganfainberg: the real problem is that you don't like crawling through dirs during the tests and what (if anything) do we do in deployments | 19:40 |
| dstanek | right? | 19:40 |
| morganfainberg | dstanek, i think crawling the directories and hoping things are named sanely is bad | 19:41 |
| ayoung | morganfainberg, that was the argument for using migrations to setup the schema for the tests. We had that a year+ ago | 19:42 |
| morganfainberg | dstanek, i would rather have models in one place vs scattered all over if we need to centralize like that | 19:42 |
| ayoung | the issue was really just speed | 19:42 |
| morganfainberg | ayoung, i can move us back to migrations, it's just slow | 19:42 |
| ayoung | but migrations are the cannonical approach | 19:42 |
| morganfainberg | ayoung, and it doesn't really buy us a lot imo | 19:42 |
| ayoung | It means we test our production code path | 19:42 |
| morganfainberg | i would almost argue we should always migrate in all extensions | 19:42 |
| morganfainberg | rather than having varying schemas | 19:43 |
| morganfainberg | ayoung, we do test that with the SQL migration tests. | 19:43 |
| ayoung | we can do that | 19:43 |
| dstanek | morganfainberg: the hard part there is that sometimes you only need catalog sql and not identity sql. does that matter? | 19:43 |
| morganfainberg | dstanek, not really | 19:43 |
| ayoung | the migrations are called by the command line. The default there if called with no params can be "migrate all exensions to their max" | 19:43 |
| morganfainberg | ayoung, no i mean, always migrate in extensions (in tree ones) even for prod | 19:44 |
| morganfainberg | ayoung, is there a reason not to add the schema information | 19:44 |
| ayoung | Ah. I think that was the origianal intention | 19:44 |
| morganfainberg | ayoung, oh ok then :) | 19:44 |
| ayoung | I'd be OK with that, too. | 19:44 |
| morganfainberg | ayoung, the second question, would you be opposed to in-tree (even extensions) keeping their models in a common module | 19:44 |
| morganfainberg | e.g. keystone.common.sql.models.<module> | 19:45 |
| ayoung | each backend should have its own migrate repo. Then again, we should treat core modules as nothing more than extensions that have been blessed | 19:45 |
| morganfainberg | so extension would have it's own module for that | 19:45 |
| morganfainberg | ayoung, each backend would keep it's own migrate repo for sure | 19:45 |
| ayoung | lets keep the migrations split | 19:45 |
| morganfainberg | ayoung, just the SQLA models themselves | 19:45 |
| ayoung | that way wehn we cut an extension we don't end up with "oops, FK" | 19:45 |
| ayoung | hmmm, I don't think I like that. | 19:46 |
| ayoung | I think I like the current approach. It seems more like the problem is figuring out which extensions have SQL models | 19:46 |
| ayoung | would you just lump "core" together, or would it require extensions as well? | 19:47 |
| morganfainberg | ayoung, i'd make assignment be "assignment_models.py" | 19:47 |
| morganfainberg | or similar | 19:47 |
| morganfainberg | same w/ identity | 19:47 |
| morganfainberg | each subsystem would be explicitly named out | 19:48 |
| ayoung | so... backends/sql/models.py ? | 19:48 |
| morganfainberg | we could do that | 19:48 |
| dstanek | what's the difference between that and what we have? | 19:48 |
| ayoung | I think that is the right format if we do. What does that buy us? | 19:48 |
| bknudson | we already have a keystone.common.sql | 19:48 |
| morganfainberg | nothing | 19:49 |
| morganfainberg | dstanek, ^ | 19:49 |
| dstanek | morganfainberg: i just want a way to "discover" the models in a reliable way | 19:49 |
| morganfainberg | it's more of a question, do we want models to be separate from the extensions (and core) in a common location or do we want to walk the tree to find them | 19:49 |
| dstanek | naming convention seemed to be that way since we seem to be good at enforcing it | 19:50 |
| morganfainberg | dstanek, i'm with you, i don't know if looking for "sql.py" is the best option | 19:50 |
| dstanek | i like them separate just because it keeps the code closer to where it's used | 19:50 |
| morganfainberg | dstanek, i think the issue here is when you're using a build system you aren't guaranteed to be running in the keystone directory | 19:51 |
| morganfainberg | you might be running elsewhere and end up with "home" or some other thing and home.keystone.thing doesn't exist | 19:51 |
| morganfainberg | or worse | 19:51 |
| dstanek | morganfainberg: why would you need to be in the keystone directory? | 19:51 |
| morganfainberg | or in nkinder's issue keystone wasn't installed in the venv and so it tried to import a non-existant model | 19:52 |
| morganfainberg | dstanek, who knows where the build system sticks files before trying do things. | 19:52 |
| morganfainberg | dstanek, or if "." is in the python path | 19:52 |
| morganfainberg | i just don't like "guessing" this is the file to import | 19:52 |
| morganfainberg | i'd rather it be a bit more rigid. | 19:53 |
| dstanek | the keystone package has to be in the path for anything to work. i think that is all my code depends on | 19:53 |
| morganfainberg | dstanek, then i'm not sure how dhellmann ended up with "home" in his path or nkinder's test run failed | 19:54 |
| morganfainberg | erm home in the module path | 19:54 |
| dstanek | it's possible then that the line that does the import needs to do a slice - i'll try think from outside of keystone | 19:54 |
| morganfainberg | dstanek, sure. | 19:55 |
| morganfainberg | dstanek, if you think the os walk is the best possible option | 19:56 |
| dstanek | morganfainberg: i don't, but that's the only one i had :-) | 19:56 |
| morganfainberg | dstanek, i'm happy to continue with it, but i think i'd rather see if we can do something better :) | 19:56 |
| morganfainberg | ok s/best possible option/current best plan/ | 19:56 |
| morganfainberg | :) | 19:56 |
| ayoung | morganfainberg, I think the sql code is explicitly part of the Driver. Spliiting it from the driver seems like splitting along the wrong axis | 19:57 |
| dstanek | morganfainberg: i don't know how else to do it besides forcing all sql module to be in one place or to make them register somewhere | 19:57 |
| morganfainberg | dstanek, perhaps registering is the right answer? | 19:57 |
| morganfainberg | ayoung, solid enough argument for not splitting it up for me | 19:58 |
| morganfainberg | dstanek, i kindof with SQLA didn't automagically register the model when you defined it on the declarative base | 19:58 |
| morganfainberg | dstanek, s/with/wish | 19:59 |
| ayoung | morganfainberg, why did dean submit the entrypoint patch? What does stevedor propose to do for us here? | 19:59 |
| morganfainberg | dstanek, we could be more programatic then. | 19:59 |
| dstanek | ayoung: in this case nothing - with can use and inspect entrypoints without it | 19:59 |
| morganfainberg | ayoung, it solves an issue wtih bad logic slicing up the name and trying to import the module to get the SQLA models when you don't use_develop (or similar) in build systems | 19:59 |
| dstanek | it just saves us a bit of code | 19:59 |
| ayoung | what is use_develop? | 20:00 |
| morganfainberg | ayoung, but it's mostly saving some code on one front | 20:00 |
| ayoung | it seems brittle? | 20:00 |
| morganfainberg | ayoung, use_develop is a python-ism to symlink to the local directory vs. instealling the stuff into the packages, it's a setuptools ism | 20:00 |
| *** jsavak has joined #openstack-keystone | 20:01 | |
| morganfainberg | ayoung, stevedore is good for loading modules consistently, but i don't know (after this discussion) if it's the option we want for this problem | 20:01 |
| morganfainberg | ayoung, i'd like to use stevedore for loading the backends, but that is a different reason than getting the models for SQL into the python namespace | 20:02 |
| dstanek | morganfainberg: what does stevedor do on top on entrypoints? does it just replace the __import__? | 20:03 |
| morganfainberg | dstanek, it replaces the __import__ and can instantiate the object directly as well w/ arguments/config | 20:03 |
| ayoung | morganfainberg, so my take is the entrypoints are like the explicitl exported symbols you get from dll_loadibrary or the elf equivalent, versus python class names being more like the native symbols you would get from the symbol table. One is more "public" than the other, but if you try hard enough you can get access to either | 20:04 |
| morganfainberg | dstanek, it's mostly a standardized framework that is a bit more straightforward than using importutils everywhere | 20:04 |
| *** joesavak has quit IRC | 20:04 | |
| dstanek | morganfainberg: i wonder if it gives us a hook to do DI | 20:04 |
| morganfainberg | dstanek, that is a good question | 20:05 |
| morganfainberg | dstanek, that'd be useful | 20:05 |
| morganfainberg | ayoung, that sounds about right. you can be more/less granular than objectclasses | 20:05 |
| dstanek | what concerns me there is the way i'd like to see DI work | 20:06 |
| morganfainberg | ayoung, it's a little more open than the exported symbols in DLLs (you could have other packages register into the same namespace) but yes. | 20:06 |
| morganfainberg | dstanek, even if we only use stevedore for specifying the driver, i think it's a win. we could even package (not saying we should) the assignment backends separate from keystone if we wanted in that case. | 20:07 |
| morganfainberg | dstanek, it may not be worth using for DI at all. | 20:08 |
| morganfainberg | dstanek, at worst is brings us more in line with other OpenStack projects on importing modules and using those imports as a driver/object. at best we can do soemthing far cooler | 20:08 |
| dstanek | that would be interesting if that were the case | 20:09 |
| morganfainberg | s/importing/dynamically importing based on config/ | 20:09 |
| *** topol has quit IRC | 20:09 | |
| morganfainberg | dstanek, so lets get back to the SQL thing :) | 20:10 |
| morganfainberg | dstanek, i like this convo, but lets solve the small issue first if we can. | 20:10 |
| *** andreaf has quit IRC | 20:11 | |
| morganfainberg | dstanek, oh i found the bug. | 20:11 |
| ayoung | Is there a "real" problem with the sql thing? | 20:11 |
| morganfainberg | ayoung, minor problem. | 20:11 |
| morganfainberg | dstanek, i see where nkinder ran afoul | 20:11 |
| dstanek | morganfainberg: what was the issue? | 20:12 |
| nkinder | what'd I do this time? :) | 20:12 |
| morganfainberg | nkinder, you did everything right and showed us a bug, | 20:12 |
| morganfainberg | dstanek, oh wait i'm wrong :( | 20:13 |
| morganfainberg | damn. | 20:13 |
| morganfainberg | somehow we end up with '' as the module name after root.replace(os.sep, '.') | 20:14 |
| morganfainberg | nkinder, you still didn't do anything different than you should have. | 20:14 |
| dstanek | morganfainberg: my guess is that root needs to have the first part ripped off base on keystone_root | 20:15 |
| dstanek | do you know how i can replicate? | 20:15 |
| morganfainberg | well for dhellmann's case, set use_develop in tox.ini to false (it looks like) | 20:16 |
| morganfainberg | i'm trying it now | 20:16 |
| morganfainberg | dstanek, and yea, looks like a slice should work | 20:17 |
| morganfainberg | stil not sure why you end up with '' as the module_name | 20:18 |
| morganfainberg | ack i gotta go, running late | 20:19 |
| morganfainberg | bbib | 20:19 |
| nkinder | ayoung: I'm testing the remember the dn patch, and I'm seeing something odd (unrelated I think/hope) | 20:21 |
| ayoung | ? | 20:21 |
| nkinder | ayoung: I run "keystone user-update --name=foo --email=bar <id>" for a valid id | 20:21 |
| nkinder | it hits ldap, searches for the user and gets a result, then searches the enabled group (using emulated enabled option) | 20:22 |
| nkinder | then I get a 404 and no mod occurs | 20:22 |
| nkinder | ayoung: create, delete, and list are working fine | 20:22 |
| nkinder | get too | 20:23 |
| ayoung | nkinder, that is way too close to our code to be coincidental | 20:23 |
| nkinder | interesting that all of the unit tests pass though | 20:23 |
| nkinder | ...though the unit tests are using fakeldap and I'm using live now | 20:23 |
| ayoung | yeah, fake is , well, fake | 20:25 |
| nkinder | ayoung: which keystoneclient commands map to the group stuff in identity? | 20:27 |
| ayoung | nkinder, um...let me see... | 20:28 |
| ayoung | none | 20:28 |
| ayoung | I thinkthat stuff was never added to CLI | 20:28 |
| ayoung | you'll have to Curl it | 20:28 |
| ayoung | nkinder, least, I don't see any group operations in the CLI args list | 20:29 |
| nkinder | ayoung: me either | 20:29 |
| nkinder | I suppose I'll need to use curl then | 20:29 |
| ayoung | I might have an example | 20:29 |
| *** bach has quit IRC | 20:29 | |
| ayoung | nope | 20:30 |
| ayoung | but this might start you on the way: http://adam.younglogic.com/2013/09/keystone-v3-api-examples/ | 20:30 |
| ayoung | you could also probably do it from python using the API directly even easier | 20:31 |
| ayoung | nkinder, https://review.openstack.org/#/c/82687/10/examples/scripts/initialize_keystone.py | 20:32 |
| ayoung | that is why we need ^^ patch | 20:32 |
| *** dstanek is now known as dstanek_zzz | 20:33 | |
| *** bach has joined #openstack-keystone | 20:33 | |
| nkinder | ayoung: yeah, I've used your example before. I can tweak it and use the group API | 20:33 |
| ayoung | ++ | 20:33 |
| *** joesavak has joined #openstack-keystone | 20:37 | |
| *** leseb has quit IRC | 20:39 | |
| *** jsavak has quit IRC | 20:41 | |
| *** leseb has joined #openstack-keystone | 20:41 | |
| *** harlowja is now known as harlowja_away | 20:44 | |
| *** dstanek_zzz is now known as dstanek | 20:50 | |
| nkinder | ayoung: that 404 error is interesting. I added a pdb.set_trace() at the top of update_user() in the LDAP driver, and it never gets hit. | 20:54 |
| nkinder | ayoung: debug logging shows that it searches and finds the user, but they it gets this: | 20:55 |
| nkinder | 2014-04-23 13:52:18.978 27998 INFO eventlet.wsgi.server [-] 127.0.0.1 - - [23/Apr/2014 13:52:18] "PUT /v3/users/141812f41b67434296b5fd6eebceedcd HTTP/1.1" 404 228 0.001869 | 20:55 |
| ayoung | nkinder, let me see... | 20:55 |
| *** leseb has quit IRC | 20:56 | |
| ayoung | nkinder, v2 or v3 update_user? | 20:56 |
| ayoung | using the cli? | 20:56 |
| nkinder | ayoung: yes, cli | 20:57 |
| *** leseb has joined #openstack-keystone | 20:57 | |
| ayoung | nkinder, is there a project in that request? There looks like a lot of project logic in the controller | 20:57 |
| nkinder | ayoung: using v3 for OS_SERVICE_ENDPOINT | 20:57 |
| bknudson | group commands are in the openstack CLI | 20:57 |
| nkinder | ayoung: I didn't explicitly set one, so it's just "default" | 20:58 |
| ayoung | bknudson, not in the help message | 20:58 |
| bknudson | openstack --os-identity-api-version=3 --os-auth-url=http://localhost:5000/v3 group create admin | 20:58 |
| bknudson | ayoung: the help message is magic and changes based on --os-identity-api-version ! | 20:58 |
| nkinder | bknudson: ok, thanks. I'll use that for testing the group side of things | 20:58 |
| nkinder | magic == smart | 20:58 |
| *** leseb has quit IRC | 20:58 | |
| bknudson | might be nicer if it said (only available with identity-api-version=3) | 20:59 |
| *** leseb has joined #openstack-keystone | 20:59 | |
| ayoung | http://openstackreactions.enovance.com/2014/04/when-looking-over-sqlalchemy-migration-code-to-see-how-it-works/ | 21:01 |
| *** Anju_ has quit IRC | 21:01 | |
| *** dstanek is now known as dstanek_zzz | 21:02 | |
| nkinder | ayoung: I like your new shirt | 21:02 |
| ayoung | I always wished I could grow my hair that long, but gravity never takes effect, so it only grows up | 21:03 |
| *** leseb has quit IRC | 21:03 | |
| *** harlowja_away is now known as harlowja | 21:05 | |
| *** erecio has quit IRC | 21:07 | |
| *** david-lyle has quit IRC | 21:09 | |
| *** andreaf has joined #openstack-keystone | 21:21 | |
| *** derek_c has joined #openstack-keystone | 21:27 | |
| *** jamielennox|away is now known as jamielennox | 21:40 | |
| *** joesavak has quit IRC | 21:41 | |
| ayoung | nkinder, is there some sort of SELinux reason that keystone couldn't make an LDAP call? | 21:43 |
| nkinder | ayoung: yes, if you're running it within httpd | 21:44 |
| *** bach has quit IRC | 21:45 | |
| ayoung | nkinder, probably same rules for eventlet then | 21:45 |
| nkinder | ayoung: not necessarily (unless it's confined) | 21:45 |
| nkinder | getsebool -a | grep ldap | 21:45 |
| nkinder | ayoung: httpd_can_connect_ldap --> off | 21:45 |
| nkinder | ayoung: user setseboot to allow it to connect | 21:46 |
| ayoung | yeah, I have that | 21:46 |
| nkinder | yeah, off means it's not allowed | 21:46 |
| *** derek_c has quit IRC | 21:46 | |
| ayoung | nope | 21:47 |
| ayoung | nkinder, I'm getting "2014-04-23 21:46:57.853 25944 TRACE keystone.common.wsgi SERVER_DOWN: {'desc': "Can't contact LDAP server"} | 21:47 |
| nkinder | ayoung: what ldap server are you using? | 21:48 |
| ayoung | i can do an ldapsearch from thesmae machine from the command line, so I know the URL is good | 21:48 |
| ayoung | its an internal IPA | 21:48 |
| nkinder | ayoung: ps -efZ and grep for keystone | 21:48 |
| nkinder | what context is it running as? | 21:48 |
| *** med_ has joined #openstack-keystone | 21:48 | |
| ayoung | system_u:system_r:keystone_t:s0 keystone 26057 1 0 21:47 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all | 21:49 |
| nkinder | ah, keystone_t | 21:49 |
| ayoung | its a packstack install, so its run from systemd | 21:49 |
| ayoung | I haven't switched this one over to running in httpd yet. One step at a time | 21:49 |
| nkinder | interesting... It would need it's own rule to connect | 21:49 |
| nkinder | ayoung: you can manually create a rule, but a boolean would be nice | 21:50 |
| nkinder | ayoung: might be worth a selinux-policy bug if you don | 21:50 |
| nkinder | don't see an existing boolean for it | 21:50 |
| nkinder | keystone_can_connect_ldap | 21:50 |
| nkinder | ayoung: do you see AVC messages? | 21:51 |
| *** bach has joined #openstack-keystone | 21:51 | |
| nkinder | ayoung: you might need to turn off dontaudit rules to be sure | 21:51 |
| nkinder | ayoung: semanage dontaudit off | 21:51 |
| mfisch | Can I get a +2 from someone on this? https://review.openstack.org/#/c/87068/ | 21:51 |
| ayoung | nkinder, yep type=AVC msg=audit(1398289617.851:119991): avc: denied { name_connect } for pid=25944 comm="keystone-all" dest=389 scontext=system_u:system_r:keystone_t:s0 tcontext=system_u:object_r:ldap_port_t:s0 tclass=tcp_socket | 21:53 |
| nkinder | ayoung: using wireshark to watch keystone->ldap is very enlightening | 21:53 |
| ayoung | what is dontaudit? | 21:53 |
| nkinder | ayoung: 8 operations (not counting binds) just to add a user to a group | 21:53 |
| nkinder | with 8 individual connections | 21:53 |
| nkinder | and ssl/tls handshakes, etc. | 21:54 |
| ayoung | nkinder, whomever added LDAP support to Keystone obviosuly had no idea what he was doing | 21:54 |
| nkinder | ayoung: counting binds and unbinds, it's 24 LDAP operations | 21:54 |
| nkinder | though unbind is essentially free | 21:54 |
| nkinder | ayoung: I'm producing a little report on the operations for user/group CRUD | 21:54 |
| nkinder | dontaudit is on by default. It causes certain AVC denials to be silent (no logging) | 21:55 |
| nkinder | it's for common cases that might occur but not be fatal | 21:55 |
| *** bach has quit IRC | 21:56 | |
| jamielennox | morganfainberg: did you ever look at jsonschema stuff? | 21:58 |
| morganfainberg | jamielennox, not in depth | 22:00 |
| ayoung | nkinder, its def selinux. setenforce 0 makes it work | 22:00 |
| morganfainberg | jamielennox, been on other tasks here (limited bandwidth last couple days) | 22:00 |
| nkinder | ayoung: yeah, I'd just add a custom policy module to allow it. It is worth a bug to ask for a boolean | 22:00 |
| ayoung | I'm going to leave it on permissive until I straighten out the database, then I'll move to keystone in httpd and re-enfroce | 22:00 |
| ayoung | enforce | 22:00 |
| *** bach has joined #openstack-keystone | 22:00 | |
| ayoung | nkinder, myabe...but I would rather drive on to httpd-ness instead | 22:00 |
| ayoung | the more work we do with the borken eventlet approach them ore pain it will be to wean ourselves. | 22:01 |
| nkinder | ayoung: but others may run into this. The selinux devs could add it very easily | 22:01 |
| ayoung | Yes they could | 22:01 |
| ayoung | but you can't polish a t*** | 22:02 |
| jamielennox | morganfainberg: i was playing with it yesterday - essentially i want to know if we could build something similar to WSME but based on jsonschema | 22:02 |
| jamielennox | morganfainberg: thought it'd be good to get someone else's opinion to see if i'm way off | 22:02 |
| morganfainberg | jamielennox, that would be cool | 22:02 |
| jamielennox | http://paste.openstack.org/show/76853/ | 22:02 |
| ayoung | nkinder, actually, this is just one of a slew of issues going to LDAP in packstack | 22:02 |
| morganfainberg | jamielennox, looking at the paste now | 22:02 |
| jamielennox | i think it's promising but it feels slightly off in implementation | 22:04 |
| *** derek_c has joined #openstack-keystone | 22:04 | |
| ayoung | I'll open the bug, though | 22:04 |
| ayoung | nkinder, what is the component that has default policy? | 22:06 |
| ayoung | is it just policy? | 22:07 |
| morganfainberg | jamielennox, hm. ah i see what you're doing | 22:08 |
| morganfainberg | took me a bit to switch into metaprogramming mindset | 22:08 |
| jamielennox | i tried to get around it | 22:08 |
| morganfainberg | sometimes you just can't | 22:08 |
| jamielennox | WSME doesn't use it, but it does some hacky things to not | 22:08 |
| jamielennox | all the metaclass should be doing is collecting class properties | 22:08 |
| morganfainberg | if it's easier to do with metaprogramming, besides the immidiate "are you sure metaprogramming is the right way" question, it's not bad to use it for clarity | 22:09 |
| morganfainberg | really hacky to avoid metaclasses is just as bad | 22:09 |
| jamielennox | i agree | 22:09 |
| ayoung | nkinder, https://bugzilla.redhat.com/show_bug.cgi?id=1090669 | 22:09 |
| uvirtbot | ayoung: Error: Could not parse XML returned by bugzilla.redhat.com: HTTP Error 404: Not Found | 22:10 |
| morganfainberg | uvirtbot, nice. | 22:10 |
| uvirtbot | morganfainberg: Error: "nice." is not a valid command. | 22:10 |
| morganfainberg | LOL | 22:10 |
| ayoung | morganfainberg, we just put on some XSRF protection, I suspect that is what is breaking it | 22:10 |
| morganfainberg | ayoung, that makes sense | 22:10 |
| ayoung | nkinder, so packstack with LDAP/IPA needs to be planned out. I talked a little with simo about setting passwords. but the user-adds are all going to have to be IPA commands, not keystone. | 22:12 |
| *** marcoemorais has quit IRC | 22:13 | |
| morganfainberg | jamielennox, so i'd be concerned with some of the choices of instance variable names in array_validator | 22:14 |
| jamielennox | morganfainberg: sure | 22:14 |
| *** harlowja has quit IRC | 22:14 | |
| bknudson | stevemar: any reason https://review.openstack.org/#/c/85213/ isn't +A? | 22:14 |
| jamielennox | most of the naming is off | 22:14 |
| *** marcoemorais has joined #openstack-keystone | 22:14 | |
| morganfainberg | jamielennox, yeah | 22:14 |
| jamielennox | morganfainberg: it just doesn't feel right that it's so based around the Model | 22:15 |
| bknudson | morganfainberg: https://review.openstack.org/#/c/85213/ ? | 22:15 |
| jamielennox | and that the Model is an object and not a Type like the others | 22:15 |
| morganfainberg | jamielennox, agree | 22:15 |
| bknudson | morganfainberg: (you had reviewed it earlier) | 22:15 |
| morganfainberg | bknudson, looking, i think this is a simple spot check for me. | 22:15 |
| morganfainberg | jamielennox, let me look at this review rally quickly | 22:16 |
| jamielennox | i don't really know a way around it because of the way the __get__ and __set__ work | 22:16 |
| jamielennox | morganfainberg: np | 22:16 |
| *** stevemar has quit IRC | 22:17 | |
| *** marcoemorais has quit IRC | 22:18 | |
| morganfainberg | bknudson, +2/+A lgtm, my concerns were addressed with the clearer message on the exception | 22:20 |
| morganfainberg | jamielennox, ok looking at the __get__ and __set__ bits | 22:20 |
| morganfainberg | jamielennox, noriced you also had the ObjectValidator commented out | 22:22 |
| jamielennox | right, have you seen the __get__ and __set__ before? | 22:22 |
| morganfainberg | jamielennox, totally aside, was that not working? | 22:22 |
| jamielennox | i don't know | 22:22 |
| morganfainberg | the descriptor special methods? | 22:22 |
| morganfainberg | i've not used __get__ and __set__ before | 22:23 |
| jamielennox | ihttp://nbviewer.ipython.org/urls/gist.github.com/ChrisBeaumont/5758381/raw/descriptor_writeup.ipynb | 22:23 |
| jamielennox | it means that you an define the property and set the value on the object | 22:23 |
| *** marcoemorais has joined #openstack-keystone | 22:24 | |
| jamielennox | so when you set m = Model() m.prop_int the data is actually being set on the m object | 22:24 |
| jamielennox | so the 'types' don't store any actual data | 22:25 |
| morganfainberg | jamielennox, yeah i see that now. this is a good writeup btw | 22:25 |
| jamielennox | but that means you have to have an actual object at the root | 22:25 |
| jamielennox | which is why the base object is not a 'type' like the others and why i comment out the object one | 22:25 |
| morganfainberg | ah | 22:26 |
| jamielennox | it means though that everything is based around the object though - you couldn't validate an array or an integer directly | 22:27 |
| morganfainberg | jamielennox, right you can't just say do_validate(thing) it needs to be passed through | 22:28 |
| morganfainberg | jamielennox, hm. | 22:28 |
| jamielennox | I was wondering if it's worth building them as actual types - but i think that's overthinking it | 22:29 |
| *** bach has quit IRC | 22:33 | |
| morganfainberg | jamielennox, yeah not sure building this as a type would be beneficial here | 22:33 |
| morganfainberg | it would be more consistent, but not exactly "better" | 22:33 |
| jamielennox | was thinking if the outcome of integer = Integer(minimum=1, maximum=5) then you could do i= integer(4) | 22:34 |
| jamielennox | because it feels like it's mixing the type of the object and the data of the object | 22:34 |
| morganfainberg | jamielennox, get full comparitor functionality | 22:34 |
| *** amcrn has quit IRC | 22:34 | |
| morganfainberg | erm, | 22:35 |
| morganfainberg | damn it | 22:35 |
| morganfainberg | brain crossed words | 22:35 |
| morganfainberg | yes | 22:35 |
| jamielennox | could do that with __call__ but i don't know what that returned object would be | 22:35 |
| *** thedodd has quit IRC | 22:36 | |
| morganfainberg | jamielennox, i like __call__ less. | 22:36 |
| jamielennox | so the __get__ __set__ make sense | 22:37 |
| morganfainberg | jamielennox, yeah | 22:37 |
| morganfainberg | jamielennox, it does. | 22:37 |
| jamielennox | because if you do i = Integer() i = 5 then you are replacing the variable - not doing a __set__ | 22:37 |
| morganfainberg | jamielennox, i think with comments, it will make more sense than using __call__ | 22:37 |
| morganfainberg | rebinding issue | 22:37 |
| morganfainberg | right | 22:37 |
| *** bach has joined #openstack-keystone | 22:38 | |
| jamielennox | and Object could be a Type, if we could have somewhere else to actually store the data | 22:38 |
| jamielennox | could just do a Validator object or something | 22:38 |
| morganfainberg | jamielennox, this argues for simply making a clever response class that acts like we want it to | 22:38 |
| morganfainberg | jamielennox, erm response/data | 22:38 |
| morganfainberg | jamielennox, which is what your model class is now. | 22:39 |
| jamielennox | morganfainberg: except that at the moment that's being used instead of an Object type | 22:39 |
| jamielennox | and it implies an object | 22:39 |
| *** derek_c has quit IRC | 22:39 | |
| jamielennox | it wouldn't let us validate an integer or anything directly | 22:39 |
| morganfainberg | jamielennox, fair. | 22:39 |
| morganfainberg | jamielennox, lets start with what points are we validating? | 22:40 |
| morganfainberg | input on the API, right? | 22:40 |
| jamielennox | the problem isn't the base one though, it's when you get nested objects | 22:40 |
| *** bach has quit IRC | 22:40 | |
| morganfainberg | ah, ahhh | 22:40 |
| jamielennox | so what i was hoping to do was provide an @input(UserRequest) and @output(UserResponse) | 22:41 |
| jamielennox | and figure out how we could hook up an Accept to return get_schema() | 22:41 |
| jamielennox | (that bit's later) | 22:42 |
| morganfainberg | hehe | 22:42 |
| jamielennox | to start with the object would have to be similar to the dict of now | 22:43 |
| morganfainberg | which is really fine | 22:43 |
| jamielennox | morganfainberg: yes and no | 22:43 |
| jamielennox | i'm not sure how you cross those object boundaries but it can be figured out | 22:44 |
| morganfainberg | this is a ton of moving parts, :( | 22:45 |
| jamielennox | right | 22:45 |
| jamielennox | this is what i mean, i don't think there is enough logical seperation between data and type | 22:45 |
| morganfainberg | yeah i can see that now | 22:45 |
| jamielennox | i was thinking about this for a day or so before i started but i didn't think it'd be this hard to keep seperate | 22:46 |
| morganfainberg | at face value this looks "easy"* | 22:46 |
| morganfainberg | * = some value of easy | 22:46 |
| jamielennox | yea, i thought it'd be mostly figuring out what parts of jsonschema to support | 22:46 |
| morganfainberg | lets step back, is there any way we can simplify this a bit | 22:47 |
| jamielennox | whether you do validation on __get__ and __set__ or on validate() | 22:47 |
| morganfainberg | bear with me, trying to step through it | 22:48 |
| morganfainberg | so we want input validation - user sends a request, we need to make sure it conforms to the correct data | 22:48 |
| jamielennox | morganfainberg: this is why i was getting someone else to bounce ideas off :) | 22:48 |
| morganfainberg | are we also aiming to do output validation "output is supposed to look like XXXX"? or mostly input | 22:49 |
| jamielennox | morganfainberg: i think input is the most obvious, if we can get that working then i thinnk we would pick up a number of bugs doing output | 22:49 |
| morganfainberg | right | 22:49 |
| jamielennox | but maybe that's a debug option | 22:49 |
| morganfainberg | just making sure, output validation sounds great for debug / tests | 22:50 |
| morganfainberg | but it could be very odd responses if output validation failed in production | 22:50 |
| morganfainberg | anyway | 22:50 |
| jamielennox | right | 22:51 |
| morganfainberg | request comes in, we go json -> python | 22:51 |
| morganfainberg | now we're at where we are today | 22:51 |
| morganfainberg | could we invert this? | 22:53 |
| morganfainberg | could we instead of going json -> object, we do validate(json) -> object | 22:53 |
| morganfainberg | jsonschema seems to support that kind of concept | 22:53 |
| morganfainberg | oh wait. nvm misreadinfg it | 22:53 |
| jamielennox | no, you start with python objects | 22:54 |
| morganfainberg | yeah. | 22:54 |
| morganfainberg | just saw that | 22:54 |
| jamielennox | which is good cause it means you can do xml as well | 22:54 |
| jamielennox | or any input that comes through to a python object we understand | 22:54 |
| morganfainberg | jamielennox, right. | 22:55 |
| morganfainberg | jamielennox, this is a tough one | 23:04 |
| jamielennox | morganfainberg: it seems though that there should be an elegant solution | 23:04 |
| jamielennox | otherwise i'd probably just have scrapped the idea and used jsonschema directly | 23:04 |
| morganfainberg | jamielennox, take the json and use something like protobuf as the validator (deseriealize it into the protobuf object) and use that for validation (only partly serious/joking) | 23:05 |
| morganfainberg | jamielennox, i mean, that is what it feels like it should be: define simple schema, take serialized data and deserialize to <object> | 23:05 |
| morganfainberg | jamielennox, ??? | 23:05 |
| morganfainberg | jamielennox, profit! | 23:05 |
| morganfainberg | deserialization should implicitly validate | 23:06 |
| nkinder | ok, the extra searches for users and groups in the ldap driver are basically just there for exceptions | 23:07 |
| *** richm has quit IRC | 23:07 | |
| nkinder | if you try to check if a user is in a group, you get one exception if they are not a member (but are a valid user) | 23:07 |
| nkinder | you get a different exception is the user doesn't exist | 23:07 |
| nkinder | is that behavior really required? | 23:08 |
| nkinder | jamielennox, morganfainberg: perhaps you guys have some thoughts on it? ^^^ | 23:09 |
| morganfainberg | nkinder, we often look for a usernotfound error vs a groupnotfound etc | 23:10 |
| morganfainberg | nkinder, we do similar things in the other backends. | 23:10 |
| morganfainberg | nkinder, or uhm. whatver the other error really is | 23:10 |
| nkinder | UserNotFound vs NotFound | 23:11 |
| morganfainberg | nkinder, one is more specific | 23:11 |
| morganfainberg | nkinder, it's useful to differentiate, but they're both 404s | 23:12 |
| nkinder | here's what sucks about how we do this... this is what we do when adding a member to a group with the ldap driver: | 23:12 |
| nkinder | search user, search enabled, search group, search user, search enabled, search group, search group, modify group | 23:12 |
| morganfainberg | nkinder, we can probably streamline that. | 23:13 |
| nkinder | around each of those, there is connect, bind, <op>, unbind | 23:13 |
| morganfainberg | nkinder, we've already searched user, so we _might_ get away with knowing we did that | 23:13 |
| nkinder | morganfainberg: yeah, I see value in the separate exceptions | 23:13 |
| morganfainberg | but isn't that the DN comparison issue? | 23:13 |
| nkinder | morganfainberg: no, this is with the DN fixes | 23:13 |
| morganfainberg | ah | 23:13 |
| nkinder | I need to see what it is without them | 23:13 |
| jamielennox | yea, there are a lot of lookups that are just validation | 23:14 |
| morganfainberg | i bet we can streamline those operations (similar to how some SQLisms are moved into joins behind the scenes) | 23:14 |
| jamielennox | i was hoping that we could put some of this onto the object so fetch user only if it's not saved on the object | 23:14 |
| morganfainberg | it's a naive approach that more closely mirrors the KVS style system | 23:14 |
| jamielennox | the request object | 23:14 |
| morganfainberg | jamielennox, that is one option | 23:14 |
| morganfainberg | jamielennox *nod* | 23:15 |
| jamielennox | that's easier with pecan where the request is available everywhere | 23:15 |
| jamielennox | but the pecan thing is harder / less desirable than i would have liked | 23:15 |
| morganfainberg | dstanek_zzz, ok i think i have a simple alternative fix for dhellmann's and nkinder's problem. it is relative/abs path issues | 23:16 |
| morganfainberg | dstanek_zzz, i think i can fix this in 2 lines... or so | 23:17 |
| morganfainberg | dstanek_zzz, but it isn't pretty still | 23:17 |
| *** andreaf has quit IRC | 23:18 | |
| *** diegows has quit IRC | 23:19 | |
| morganfainberg | dstanek_zzz, actually 1 line, just a slice, but easy to duplicate | 23:19 |
| *** marcoemorais has quit IRC | 23:21 | |
| *** marcoemorais has joined #openstack-keystone | 23:22 | |
| *** gokrokve has quit IRC | 23:23 | |
| morganfainberg | nkinder, in short, i think we could make the driver a lot smarter without changing much else. | 23:24 |
| morganfainberg | nkinder, and know if we need to do the lookups or not depending on operations. | 23:24 |
| nkinder | morganfainberg: yeah. I can eliminate a good chunk of them without changing the exceptions | 23:25 |
| nkinder | morganfainberg: I think the only unavoidable one is when checking if a user is a member of a group | 23:25 |
| morganfainberg | nkinder, might even be able to work around that one. | 23:26 |
| morganfainberg | nkinder but it might require another method that is called that bypasses the extra lookup(s) in some cases. | 23:26 |
| nkinder | morganfainberg: we can iterate on it. I'll tackle what is straight-forward right now and will compare before/after | 23:27 |
| morganfainberg | nkinder, ++ sounds good | 23:27 |
| jamielennox | morganfainberg: that descriptors tutorial is really good - thats not the ne i was originally reading and came to most of those conclusions seperately | 23:30 |
| morganfainberg | jamielennox, yeah it's very good | 23:31 |
| morganfainberg | jamielennox, easy to read and understand | 23:31 |
| jamielennox | which is good that it's a standard pattern - just should have found that first | 23:31 |
| morganfainberg | hwhew | 23:31 |
| morganfainberg | hehe* | 23:31 |
| *** lbragstad has quit IRC | 23:47 | |
| *** gokrokve has joined #openstack-keystone | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!