Tuesday, 2016-04-19

*** dan_nguyen has joined #openstack-keystone00:03
*** neophy has quit IRC00:03
*** BjoernT has quit IRC00:09
openstackgerritMerged openstack/keystone: Typo in sysctl command example Edit  https://review.openstack.org/30700800:12
*** dan_nguyen has left #openstack-keystone00:14
*** dave-mccowan has joined #openstack-keystone00:14
*** dan_nguyen has joined #openstack-keystone00:14
*** mylu has joined #openstack-keystone00:15
*** spzala has quit IRC00:18
*** spzala has joined #openstack-keystone00:19
*** spzala has quit IRC00:24
*** daemontool has quit IRC00:27
*** raildo is now known as raildo-afk00:33
*** timonwong has joined #openstack-keystone00:34
mfischmorgan: fernet keys in the db?00:35
morganmfisch: as an option to make fernet default00:36
morgansince db is shared.00:36
morgani don't like it00:36
morganbut we have to consider it00:36
mfischplease no00:36
mfischI'll reply to the ml00:36
morgancome talk at the summit00:36
morganwe'll hash out the details of how we make fernet default in keystone00:36
dstanekmorgan: i'm curious to hear why we'd go down that route00:37
morgandstanek: its a question of how to have "sane"-ish defaults for fernet considering there is an operational overhead to create/sync00:38
morgandstanek: and how does keystone respond when fernet keys don't exist on $host$00:38
morganif it's the default token provider00:38
morganand i think keystone server running fernet_setup and dumping keys on disk is a terribad idea00:39
* morgan would be ok with fernet_setup being in the DB if it was protected somehow to prevent keystone server from writing00:39
morganbut could be synchronised via galera00:40
dstanekmorgan: have the issues been captured in the etherpad for the session00:40
morganbut anywah00:40
morganit's just one thing we can't say "absolutely not" unless we highlight it as an option to solve the problem00:40
morganfwiw, i dislike keys in the db00:40
morganin fact, i am certain as long as uuid and fernet are validated the same way (different is .decrypt() or .query() from DB for the payload) I am content to keep both and keep uuid as the default in keystone00:41
morganbut still one path of "validation"00:41
* mfisch wants morgan to stop moving his cheese00:43
*** tqtran has quit IRC00:44
mfischwill be a good convo at the summit in all seriousness00:44
*** gyee has quit IRC00:44
mfischokay im going to go play swbf00:44
*** roxanagh_ has joined #openstack-keystone00:45
*** roxanagh_ has quit IRC00:49
*** itlinux has quit IRC00:55
morganmfisch: i expect the answer is going to be very easonable00:58
morganmfisch: also.. YOU NO CAN HAZ CHEESE00:58
morganstevemar: BREAK THE WORLD01:07
morganstevemar: WATCH IT BURN!01:07
morganstevemar: no more CLI01:07
*** ayoung has joined #openstack-keystone01:12
*** ChanServ sets mode: +v ayoung01:12
ayoungbknudson, https://review.openstack.org/#/c/306681/1  should I back off on always testing for UUID?  Do we really plan on allowing non-uuid project IDs?01:14
patchbotayoung: patch 306681 - keystone - Make all fixture project_ids into uuids01:14
*** lhcheng has quit IRC01:14
morganayoung: i think with ldap assignment dead we can enforce uuids?01:17
morganayoung: or is there a v2 path to get non uuid in?01:17
*** spzala has joined #openstack-keystone01:19
ayoungmorgan, I hear rumors of Zombie LDAP01:20
morgani am fairly certain v3 *only* allows uuid. but........01:20
ayoungmorgan, he says that since we allow pluggable drivers, we could have a non uuid project id01:20
morganwe can say all ids should be uuid.01:21
morganbut we might have legacy to support01:21
*** edmondsw has quit IRC01:21
ayoungmorgan, I'm OK if that is the case, just don't want code reviews languishing due to uuid vs non... bknudson often makes many comments in a review, and sometimes I am not sure if they are ones he is holding firm on01:21
morgani'd probably make it explicitly deprecated that we support non-uuid ids01:21
morganfor projects01:21
ayoungif legacy then no Fernet for you!@01:21
*** EinstCrazy has joined #openstack-keystone01:22
*** spzala has quit IRC01:25
*** alejandrito has joined #openstack-keystone01:26
*** mylu has quit IRC01:29
*** csoukup has joined #openstack-keystone01:29
ayoungmorgan, whadaya say?  UUID only?  I would say that if we don't do UUID only we are going to have to specify what are leval charaters for a project Id01:31
ayoungDamned either way01:31
*** stingaci has quit IRC01:32
*** stingaci has joined #openstack-keystone01:32
*** spzala has joined #openstack-keystone01:33
morgani think we can say non-uuid is deprecated. but ... ick01:33
*** mylu has joined #openstack-keystone01:33
morgani think we would need a legacy fernet formattr01:33
morganthat supports non-uuid project ids..01:33
*** csoukup has quit IRC01:33
morganbecause i'd like to move uuid to validate the same way as fernert01:34
*** stingaci has quit IRC01:37
*** stingaci has joined #openstack-keystone01:37
ayoungNo we don't01:40
ayoungnon UUID would never have worked with Fernet01:40
ayoungmorgan, Fernet is coded to only allow UUID based projects.  THat is the problem I am trying to work out01:40
ayoungits why we can't go to Fernet defautl yet01:40
ayoungall of the unit tests do "FOO" and "BAR" type IDs01:41
*** browne has quit IRC01:41
ayoungyou mean a non UUID-project-ID uuid token provider01:41
morganfernet formaters can be any data differences01:42
ayoungmorgan, I'm Not even sure why lbragstad was looking at the project ID specifically.01:42
ayoungis it a length thing?01:42
morganbecause fernet converts to 14bytes binary01:42
morganso it is a length issue01:43
ayoungit might be Fernet is looking at UUID.01:43
ayounger Userid01:43
ayoungthat ain't gonna fly01:44
ayoungBreaks LDAP01:44
morganso we would need another formatter :(01:44
morgannot too terrible01:44
ayoungno, I mean  I wonder if Fernet works with actualy LDAP today>?01:44
ayoungIt must...01:45
ayoungI think this is the line that was a problem http://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/providers/fernet/token_formatters.py#n42701:45
ayoungmorgan, did you see the SAML question on the user list?01:49
morganayoung: yeah i saw a post01:49
ayoungI edited my response.01:49
morgani'd need to read more in depth01:49
ayoungIt origianally had "Send me your credit card number" in it01:49
ayoungThey Need ECP01:50
morganECP is important01:50
ayoungmorgan, I had An idea that I think you will like.01:51
ayoungIts about X50901:51
ayoungYou know how I was pushing Certmonger?01:51
ayoungAnd Certmonger has a self signed, but that doesn't reallly help for multi node?01:51
ayoungSo..we can, I think. hack a script into Certmonger that essentially uses SSH to call to another Certmonger01:52
ayoungso, we treat the certmonger on the controller like a CA01:52
ayoungthe getcert call is 2 parts01:52
ayoung1 is all the Database and CSR generation01:52
ayoungso compute certmonger still does that01:52
morgani'll need gto think about that01:52
ayoungthen, where certmonger would usually do a call to a provider like01:52
*** itlinux has joined #openstack-keystone01:53
stevemarmorgan: i'm with mfisch on this one :)01:53
ayounginstead is would do, in essence  ssh user@controller /usr/libexec/certmonger/local-submit01:54
ayoungthere is a little more to it, it needs to hande some env vars01:54
*** stingaci has quit IRC01:54
ayoungbut it means that we could have a consistant interface for cert management, from "selfsigned" through huge real CA01:54
ayoungstevemar, what one?01:55
morganno db fernet keys01:55
ayoungmorgan, does the library care where the keys live?01:59
morganhaven't looked01:59
ayoungIdeally, keys get generated in a container and never leave the container01:59
morgani don't think so.01:59
ayoungI'm sure they don't01:59
morgananyway. it's a convo to be had at the sumit01:59
ayoungWe have keys on disk01:59
*** EinstCrazy has quit IRC02:00
ayoungNow we have them in a database, it just makes the surface different02:00
morgandiffernet reasons02:00
ayoungI don't lkike it02:00
ayounglike it02:00
morganwill discuss at the summit02:00
ayoungit means that the key is in something inhernatly remotable02:00
morganit's about steps to make it the default in keystone02:00
morganand i don;t see that as viable02:00
*** edtubill has joined #openstack-keystone02:00
morgandue to ops overhead02:01
ayoungDB is not viable?02:01
ayoungor stored on disk is not viable?02:01
morganno ferner as ddfault with keys on disk02:01
ayoungTokens are stupid, can we just drop them?02:01
stevemari think saying uuid is default, and fernet is not supported as a default (due to extra setup), is a perfectly fine statement02:01
ayoungI feel like I've wasted half a decade trying to polish this particular ....02:01
morganfix the separation of service to service and user to service02:02
morganthen we can look at a path away from tokens02:02
ayoungstevemar, nah02:02
morganbut until we "fix" that we can't02:02
*** EinstCrazy has joined #openstack-keystone02:02
ayounguuid needs to die02:02
ayoungthe database.02:02
ayounglook, the Key DB is not that big a deal02:02
ayoungis one directory, one set of perms02:03
morganand you make it the default and you break current deployments02:03
ayoungshouldn't break anything02:03
morganif fernet keys don't exist02:03
morganand they're using the default02:03
morganwhich is uuid02:03
morgankeystone suddenlyndoesn't work anymore02:03
ayoungso maybe we go the "create on demand" route02:04
ayoungits probably fine02:04
morgannow you have an issue02:04
morgani have a cluster of keystones02:04
morganand the disks are disparate02:04
ayoungKeystone is acluster...02:04
morganand now each keystone has a different group of keys generated on demand02:04
*** itlinux has quit IRC02:04
morganand again, broken02:04
ayoungI see where you ended up with Database02:04
morganas a path02:04
morgani figure we can disucss the detauls at the summit better than on irc02:04
*** itlinux has joined #openstack-keystone02:05
lbragstadyeah - this is going to be an interesting discussion02:05
morganit's a transition from tokens that don't use keys to ones that do02:05
morganit's challenging02:05
ayoungSo long as no one proposes a custom mechanism for sharing symmetric keys02:05
morganalso, like i said moving to where user-> service and service->service is separate02:05
morganwe can move away from tokens as the long term pth02:06
ayoung3 years later and we finally have a call for Kite02:06
morganayoung: i have an alternative, but i think we can propose it bettr (and pick it apart) while at the summit02:08
*** woodster_ has quit IRC02:08
morganit'll be a good convo for sure02:08
*** stingaci has joined #openstack-keystone02:09
ayoungmorgan, the more I see these things, the more I realize just how dependent we are going to be on Heat, Tripleo, and Puppet.02:09
openstackgerritRon De Rose proposed openstack/keystone: Fixes incorrect deprecation warning for IdentityDriverV8  https://review.openstack.org/30530102:11
*** stingaci_ has joined #openstack-keystone02:12
*** itlinux has quit IRC02:12
*** itlinux has joined #openstack-keystone02:13
*** stingaci has quit IRC02:13
*** itlinux has quit IRC02:15
ayoungSo rcrit is working on autoregistration of VMs in IPA.  I think we are going to link that in to the conversation on VM-Identity/Service Users02:15
*** rock has joined #openstack-keystone02:19
*** ninag has quit IRC02:19
*** phalmos has joined #openstack-keystone02:20
*** dan_nguyen has quit IRC02:30
*** phalmos has quit IRC02:30
*** edtubill has quit IRC02:33
*** rock has quit IRC02:35
*** EinstCrazy has quit IRC02:37
*** edtubill has joined #openstack-keystone02:37
*** maxabidi has quit IRC02:39
*** browne has joined #openstack-keystone02:39
*** phalmos has joined #openstack-keystone02:40
*** alejandrito has quit IRC02:41
*** edtubill has quit IRC02:42
openstackgerritRon De Rose proposed openstack/keystone: Move the assignment abstract base class out of core  https://review.openstack.org/29963502:42
*** lhcheng has joined #openstack-keystone02:44
*** ChanServ sets mode: +v lhcheng02:44
*** lhcheng_ has joined #openstack-keystone02:45
stevemarmorgan: we need to doc https://review.openstack.org/#/c/288216/10 a bit better02:46
patchbotstevemar: patch 288216 - keystone - Customize config file location when run as wsgi app.02:46
openstackgerritRon De Rose proposed openstack/keystone: Move the resource abstract base class out of core  https://review.openstack.org/30282602:46
stevemarmorgan: sorry, that seemed random... i was going to +2 it, but thought that it needed better docs02:48
morgani giot it02:48
*** lhcheng has quit IRC02:49
stevemar+2 anyway02:50
stevemaras it'll help with the gunicorn case02:50
morganthis will break btw.02:50
morganand need rebasing02:50
morgancause it conflicts with eventlet removal02:51
*** EinstCrazy has joined #openstack-keystone02:53
morganand i think i'd rather rebase tis on eventlet remova because your patch is ...02:54
*** EinstCra_ has joined #openstack-keystone02:54
*** EinstCrazy has quit IRC02:54
*** dave-mccowan has quit IRC03:00
*** spzala has quit IRC03:00
*** spzala has joined #openstack-keystone03:01
*** dan_nguyen has joined #openstack-keystone03:04
*** spzala has quit IRC03:05
*** mylu has quit IRC03:06
*** stingaci_ has quit IRC03:06
stevemarmorgan: oh i know, my patch is gating, so yeah, it'll need a rebase03:09
stevemarmorgan: just putting my +2 on the record03:09
*** rderose has quit IRC03:16
*** tqtran has joined #openstack-keystone03:17
*** phalmos has quit IRC03:18
*** mylu has joined #openstack-keystone03:22
*** TxGVNN has joined #openstack-keystone03:23
openstackgerritMerged openstack/keystone: Fix confusing naming in ldap EnableEmuMixin.  https://review.openstack.org/30683803:24
*** roxanagh_ has joined #openstack-keystone03:35
*** hugokuo has quit IRC03:39
*** charz_ has quit IRC03:39
*** links has joined #openstack-keystone03:41
*** itlinux has joined #openstack-keystone03:41
*** hugokuo has joined #openstack-keystone03:41
*** charz has joined #openstack-keystone03:42
*** ianw has quit IRC03:43
*** itlinux has quit IRC03:45
*** itlinux has joined #openstack-keystone03:46
*** richm has quit IRC03:56
openstackgerritSteve Martinelli proposed openstack/keystone: remove fallback to default domain id  https://review.openstack.org/29482204:01
stevemarmorgan: tossed up https://review.openstack.org/#/c/294822/04:01
patchbotstevemar: patch 294822 - keystone - remove fallback to default domain id04:01
morganstevemar: looking04:01
morganstevemar: fix commit message: removed-as-of-newton BP?04:03
morganstevemar: but +204:03
stevemarmorgan: derp04:04
openstackgerritSteve Martinelli proposed openstack/keystone: update deprecation warning for falling back to default domain  https://review.openstack.org/29482204:06
stevemarthere we go04:06
*** ianw has joined #openstack-keystone04:08
*** sheel has joined #openstack-keystone04:09
*** ayoung has quit IRC04:11
*** dan_nguyen has quit IRC04:17
*** tqtran has quit IRC04:20
*** timonwong has quit IRC04:27
*** ianw has quit IRC04:27
*** pumarani- has quit IRC04:31
*** pumaranikar has joined #openstack-keystone04:32
*** ianw has joined #openstack-keystone04:33
*** zqfan has joined #openstack-keystone04:42
*** roxanagh_ has quit IRC04:44
*** spzala has joined #openstack-keystone05:01
*** mylu has quit IRC05:02
*** Nirupama has joined #openstack-keystone05:03
*** mylu has joined #openstack-keystone05:05
*** spzala has quit IRC05:06
*** rcernin has joined #openstack-keystone05:10
*** chlong has quit IRC05:16
*** timonwong has joined #openstack-keystone05:25
*** mylu has quit IRC05:30
stevemarjamielennox: do you recall the magic combination of auth_uri/url and identity_uri that was needed in liberty? https://bugs.launchpad.net/nova/+bug/1550449/comments/205:45
openstackLaunchpad bug 1550449 in python-keystoneclient "Can not create instance - liberty - centos 7" [Undecided,New]05:45
*** e0ne has joined #openstack-keystone05:54
stevemarmorgan: we've got 3 bugs that are somewhat related: https://bugs.launchpad.net/oslo.policy/+bug/1547684 + https://bugs.launchpad.net/oslo.policy/+bug/1459884 + https://bugs.launchpad.net/keystone/+bug/157187505:56
openstackLaunchpad bug 1547684 in oslo.policy "Attribute error on Token object when using domain scoped token" [Undecided,New]05:56
openstackLaunchpad bug 1459884 in oslo.policy "OR rules fail if clause throws and exception" [Undecided,Confirmed]05:56
openstackLaunchpad bug 1571875 in OpenStack Identity (keystone) "Domain role hidden by project role" [Undecided,New]05:56
*** furface has quit IRC05:56
*** spzala has joined #openstack-keystone06:02
*** e0ne has quit IRC06:05
*** spzala has quit IRC06:07
*** e0ne has joined #openstack-keystone06:10
*** roxanagh_ has joined #openstack-keystone06:24
*** roxanagh_ has quit IRC06:29
*** furface has joined #openstack-keystone06:29
*** e0ne has quit IRC06:30
*** e0ne has joined #openstack-keystone06:32
openstackgerritMerged openstack/keystone: Default caching to on for request-local caching.  https://review.openstack.org/27719806:33
*** furface has quit IRC06:36
*** jaosorior has joined #openstack-keystone06:37
*** lhcheng_ has quit IRC06:41
*** dmellado_ is now known as dmellado06:44
*** e0ne has quit IRC06:47
*** fawadkhaliq has joined #openstack-keystone06:49
openstackgerritNavid Pustchi proposed openstack/keystoneauth: Fixing D301 docstring.  https://review.openstack.org/30758706:49
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Imported Translations from Zanata  https://review.openstack.org/30758906:54
*** daemontool has joined #openstack-keystone06:55
openstackgerritRyosuke Mizuno proposed openstack/keystone: Add migration to make service type unique  https://review.openstack.org/30759306:56
openstackgerritNavid Pustchi proposed openstack/keystoneauth: Removing D211 in tox  https://review.openstack.org/30759707:02
*** spzala has joined #openstack-keystone07:03
*** browne has quit IRC07:08
*** spzala has quit IRC07:08
*** jed56 has joined #openstack-keystone07:12
*** fawadkhaliq has quit IRC07:12
*** pcaruana has joined #openstack-keystone07:16
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Updating sample configuration file  https://review.openstack.org/30760607:19
*** pcaruana has quit IRC07:23
*** tesseract has joined #openstack-keystone07:24
*** tesseract is now known as Guest2294507:24
*** roxanagh_ has joined #openstack-keystone07:26
*** jaosorior has quit IRC07:27
openstackgerritMerged openstack/keystone: Remove eventlet support  https://review.openstack.org/24948607:29
*** roxanagh_ has quit IRC07:31
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Updating sample configuration file  https://review.openstack.org/30760607:32
*** pnavarro has joined #openstack-keystone07:41
*** mariusv has quit IRC07:48
*** jaosorior has joined #openstack-keystone07:55
*** spzala has joined #openstack-keystone08:04
*** pumaranikar has quit IRC08:07
*** spzala has quit IRC08:09
openstackgerritDavanum Srinivas (dims) proposed openstack/keystone: [WIP] Testing latest u-c  https://review.openstack.org/30684808:10
*** henrynash has quit IRC08:12
*** pumaranikar has joined #openstack-keystone08:12
*** e0ne has joined #openstack-keystone08:13
*** jistr has joined #openstack-keystone08:16
*** mhickey has joined #openstack-keystone08:22
openstackgerritNavid Pustchi proposed openstack/keystoneauth: Fixing D204, D205, D208, and D211 pep8  https://review.openstack.org/30759708:36
*** sheel has quit IRC08:55
*** spzala has joined #openstack-keystone09:05
*** jaosorior has quit IRC09:07
*** spzala has quit IRC09:10
*** roxanagh_ has joined #openstack-keystone09:13
*** roxanagh_ has quit IRC09:18
*** henrynash has joined #openstack-keystone09:18
*** ChanServ sets mode: +v henrynash09:18
*** timonwong_ has joined #openstack-keystone09:20
*** hogepodge has joined #openstack-keystone09:23
*** timonwong has quit IRC09:23
*** spzala has joined #openstack-keystone10:07
*** spzala has quit IRC10:11
*** EinstCrazy has joined #openstack-keystone10:15
*** EinstCra_ has quit IRC10:18
*** EinstCrazy has quit IRC10:20
*** henrynash has quit IRC10:25
*** pnavarro is now known as pnavarro|mtg10:28
*** markvoelker has joined #openstack-keystone10:37
*** markvoelker has quit IRC10:42
*** jaosorior has joined #openstack-keystone10:42
*** timonwong_ has quit IRC10:44
*** roxanagh_ has joined #openstack-keystone11:01
*** roxanagh_ has quit IRC11:06
*** real56 has joined #openstack-keystone11:07
*** spzala has joined #openstack-keystone11:08
*** spzala has quit IRC11:13
*** doug-fish has joined #openstack-keystone11:30
*** zqfan has quit IRC11:32
*** aimeeU has joined #openstack-keystone11:35
*** markvoelker has joined #openstack-keystone11:38
*** pnavarro|mtg is now known as pnavarro11:38
*** markvoelker has quit IRC11:42
*** gordc has joined #openstack-keystone11:48
*** Guest22945 is now known as tesseract11:54
*** tesseract is now known as Guest7339711:54
*** rodrigods has quit IRC11:54
*** rodrigods has joined #openstack-keystone11:55
*** raildo-afk is now known as raildo11:58
openstackgerritSamuel de Medeiros Queiroz proposed openstack/keystone: Remove support for generating ssl certs  https://review.openstack.org/30679511:58
openstackgerritSamuel de Medeiros Queiroz proposed openstack/keystone: Remove comments mentioning eventlet  https://review.openstack.org/30740911:59
*** spzala has joined #openstack-keystone12:09
*** trown|outtypewww is now known as trown12:12
*** spzala has quit IRC12:13
*** EinstCrazy has joined #openstack-keystone12:13
*** henrynash has joined #openstack-keystone12:14
*** ChanServ sets mode: +v henrynash12:14
*** mtreinish has quit IRC12:18
*** markvoelker has joined #openstack-keystone12:19
*** spzala has joined #openstack-keystone12:19
*** mtreinish has joined #openstack-keystone12:21
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Updated from global requirements  https://review.openstack.org/30518712:24
openstackgerritOpenStack Proposal Bot proposed openstack/keystoneauth: Updated from global requirements  https://review.openstack.org/30775312:24
openstackgerritOpenStack Proposal Bot proposed openstack/keystonemiddleware: Updated from global requirements  https://review.openstack.org/30775412:24
*** henrynash has quit IRC12:28
openstackgerritOpenStack Proposal Bot proposed openstack/python-keystoneclient: Updated from global requirements  https://review.openstack.org/30777112:28
openstackgerritOpenStack Proposal Bot proposed openstack/python-keystoneclient-kerberos: Updated from global requirements  https://review.openstack.org/30777212:28
*** henrynash has joined #openstack-keystone12:45
*** ChanServ sets mode: +v henrynash12:45
*** dave-mccowan has joined #openstack-keystone12:54
*** richm has joined #openstack-keystone13:08
*** trown is now known as trown|brb13:25
*** mylu has joined #openstack-keystone13:26
*** BjoernT has joined #openstack-keystone13:26
openstackgerritSteve Martinelli proposed openstack/keystone: add missing deprecation reason for eventlet option  https://review.openstack.org/30781413:29
*** BigWillie has joined #openstack-keystone13:31
*** trown|brb is now known as trown13:34
*** openstackstatus has joined #openstack-keystone13:37
*** ChanServ sets mode: +v openstackstatus13:37
-openstackstatus- NOTICE: We have recovered one of our cloud providers, but there is a huge backlog of jobs to process. Please have patience until your jobs are processed13:40
openstackgerritBrant Knudson proposed openstack/keystone: Define identity interface - easy cases  https://review.openstack.org/29195013:42
openstackgerritBrant Knudson proposed openstack/keystone: Tests clean up global ldap settings  https://review.openstack.org/30433713:43
openstackgerritBrant Knudson proposed openstack/keystone: Opportunistic LDAP testing  https://review.openstack.org/30023713:43
*** ayoung has joined #openstack-keystone13:50
*** ChanServ sets mode: +v ayoung13:50
*** Nirupama has quit IRC13:51
*** pushkaru has joined #openstack-keystone13:52
*** henrynash has quit IRC14:00
*** sigmavirus24_awa is now known as sigmavirus2414:01
*** links has quit IRC14:03
morganstevemar: nice bugs. :(14:04
*** mylu has quit IRC14:05
*** mylu has joined #openstack-keystone14:07
*** gagehugo has joined #openstack-keystone14:07
*** henrynash has joined #openstack-keystone14:12
*** ChanServ sets mode: +v henrynash14:12
ayoungmorgan, lbragstad so, on Fernet, assuming for the moment that we cannot convert the "default" to be Fernet (yet) we need to state that the current level of Fernet testing has not been sufficient ot flush out all of the bugs.  How do we close that gap?14:13
morganayoung: default in devstack is doable14:14
ayoungI don't, for now, care that the Fernet is not default.  But I do care that things break with Fernet.  Need to  be able to honestly support it.14:14
ayoungmorgan, that is a start, but it still does not exercize the broken unit tests14:14
morganayoung: default in keystone config setting is not because of the open convos.14:14
ayoungmorgan, accepted14:15
openstackgerritSteve Martinelli proposed openstack/keystone: update deprecation warning for falling back to default domain  https://review.openstack.org/29482214:15
ayoungI think we need to rework our tests to make sure that Fernet is run through all the tests.14:15
stevemarhenrynash: better? https://review.openstack.org/#/c/294822/14:15
patchbotstevemar: patch 294822 - keystone - update deprecation warning for falling back to def...14:15
ayoungAnd, with UUID also staying as default, we need to continue to test that to the same degree14:15
morganI think we make it so uuid and fernet validate the same way. Hold the same data14:15
morganAnd then keep talking about the fernet key specific issues.14:16
lbragstadayoung ++14:16
lbragstadI also agree with morgan14:16
morganWe should reduce validation of tokens to a single code path, regardless of the provider14:16
lbragstadwe should make it so they use as much of the same code path as possible14:16
morganWith the difference being decrypt() or db query14:16
ayoungmorgan, that, too14:16
lbragstadthat will make testing easier14:16
morganThat will reduce the gap to lookup token14:17
morganAnd I think that hits your concerns14:17
morganFernet keys and default in keystone itself is a bigger question set14:17
ayoungmorgan, excpet that is a lot more work.14:17
henrynashstevemar: perfic!14:17
morganayoung: it is a lot less than you think ;)14:18
ayoungGetting Fernet fully supportable in Newton is priority14:18
morganDoable in Newton.14:18
ayoungmorgan, I've been elbows deep in this code.  Do you thin kanyone has a clearer view of what it would take? Maybe lbragstad ....14:18
*** edtubill has joined #openstack-keystone14:18
morganSec let me plugin laptop and phone.14:19
ayoungmorgan, in Newton, yes.   My goal was to have the tests passing by the summit and to have it merged in N114:19
*** woodster_ has joined #openstack-keystone14:19
morganI think I could put together a patch today.14:19
ayoungmorgan, heh...14:19
morganIf you focus on trust etc tests.14:19
lbragstadayoung do we have a recent run that showcases the latest failures?14:19
ayounglbragstad, so I have the tests passing, with code changes, module Python3 and tempest14:20
ayoungwith Py3 we need to deal with the UUID parsing...as that is meaning to_bytes is not getting called.14:20
patchbotayoung: patch 258650 - keystone - [WIP]Make fernet default token provider14:21
ayoungbut that currently skips all caching.  Validation and caching are going to be problematic until we start invallidating the cache much more aggressively14:21
morganok ayoung14:22
ayoungreally need to invalidate on all assignment changes.  It might mess with some people's tests14:22
*** jaugustine has joined #openstack-keystone14:22
morganso, the way to make these work the same is overhauling uuid driver.14:22
morganand making it basically use fernet but instead of encrypt, store the data in the DB.14:22
morganthe exact same fernet payload14:23
ayoungmorgan, morgan yeah...really, we need to chop out all data from the persisted store that would not be in the signed body of the Fernet payload14:23
morganit involves a db migration to fix the token table (ick)14:23
morganayoung: but i bet that is a 1 day task14:23
ayoungmorgan, and people store stuff in extras, right?14:23
morgannot in tokens14:23
ayoungthat makes it easier14:23
morganthere is *no* way to store extra garbage in tokens14:23
ayoungthat was the part I was dreading...whew14:24
morganshort of a custom provider14:24
lbragstadwhy can't we just nuke the info when we get it back from the DB?14:24
ayoungOK, so here is the behavioral difference14:24
morganand the way i was going to do it is change the entrypoint14:24
ayounglets say a user has 2 roles, and loses one of them14:24
morganleave the old provider in place as a stub deprecated14:24
morganand i actually was going to make a new DB table14:24
ayoungbut they first got the token when they had both.  Now when they validate, instead of the token being invalid, it needs to be valid but only have a single role on it14:24
morganto avoid the issues of touching a giant ball of ick14:24
ayounga lot of that WIP patch is test changes to deal with it14:24
morganand limit migration times.14:24
morganayoung: fernet supports more than one role14:25
morganif assignments change14:25
ayoungmorgan, when migratin the token table, we've truncated in thepast.14:25
morganassignments change14:25
lbragstadwe could technically make it so that when we return the token - we could only return the stuff that fernet would https://github.com/openstack/keystone/blob/23bb657369292cab3203c046a0a186df89fa1576/keystone/token/persistence/backends/sql.py#L9314:25
ayounglbragstad, exactly14:25
morganayoung: for deprecation purposes, i wouldn't truncate14:25
morgani would maintain uuid-legacy14:26
ayoungmorgan, your call.14:26
lbragstadthe common.py provider logic would be forced to dynamically generate everything14:26
*** csoukup has joined #openstack-keystone14:26
morganfor a deprecation cycle14:26
morganthats all14:26
morganjust to be as *nice* as possible to deployers who have custom providers14:26
ayoungso we have 3 tokens providres for a cycle.  New UUID insists on revocating events.14:26
morganand next cycle old UUID-legacy is dropped14:26
morganwe then have a single token validation path14:26
ayoungOK...the real issue is the testing matric14:27
morganit hits all your major concerns while leaving us free to work on fernet keys etc and those challenges14:27
ayoungwe need to ensure that all of the tests are run with each of the providers.  We were lax on that up to now14:27
morganbasically the old UUID tests are mothballed (left as is)14:27
morganand drop once we drop legacy14:27
morganwork on making tests for fernet/uuid-new solid14:28
morgandon't be la there14:28
morganbut since we reduced down to a single path, we're good.14:28
*** nbloom has joined #openstack-keystone14:28
morganand it also means performance work/improvements affect both forms.14:29
ayoungmorgan, can we provide a keystone-manage migrate-tokens call for when we switch from old-uuid to new....or...we'll end up dumping all tokens when we switch the default14:30
morganayoung: no.14:30
morganwell sure14:30
morganbut it wont be recommended ever14:30
ayoungmorgan, that is going to cause operator pain.  How can we avoid that14:30
morgantoken tables tend to be huge14:30
morganbut yeah i can add a migrate option14:31
morganwith a giant OMG DONT DO THIS14:31
bknudsonseems easy enough to support both formats and new tokens get the new format.14:31
nbloomHi all, I'm trying to run devstack ./stack.sh and it fails. I get "Exception occurred processing WSGI script '/usr/local/bin/keystone-wsgi-admin'" under /opt/stack/logs/.. can anyone help me? thanks14:32
ayoungdo we really need to dump the old tokens?  THe old table will have all the data we need to validate, we just will only use a subset.14:32
lbragstadayoung that's all stuff we can do in code too14:32
ayoungmorgan, let me take a look at the table format.  I think everything is in a serialized JSON blob at the moment14:33
morganayoung: it isn't14:33
morganayoung: not really. and the token table has a lot of extra *stuff* in it.14:33
morganbasically, i don't want to have to open the token and guess what the format is14:33
lbragstadit's pretty much the entire auth response14:33
lbragstadjust shoved into the extras column14:33
morganis this a legacy blob, or a fernet payload, or ???14:33
ayounghold on...I'll give a rel answer in a second14:34
morgani also don't want to "fix" the legacy provider to support the new format/ignore it in the case someone swaps back over.14:34
morganbasically, keeping them isolated during deprecation is a lot less code/work level of concern14:35
morganless testing to write too.14:35
*** slberger has joined #openstack-keystone14:35
*** sheel has joined #openstack-keystone14:36
morganso in short, yes we can just use the current table14:37
morganbut we shouldn't14:38
ayounghttp://paste.openstack.org/show/494649/  OK that is the current table format14:40
morganay yes14:40
morganayoung: yes*14:40
ayoungif we got to unified dleegation, we could just drop the valid and extra columns14:40
morganexcept unified delecation isn't landing that soon. i expect N2 or just past N214:41
ayoungmorgan, you are an optimist, but yes14:41
morgan*and* guessing at the body of the response means a lot more ick in uuid-legacy14:41
morganand a lot more complexity of code14:41
*** nbloom is now known as nbloom214:42
morganif we are looking at the short path to unified token validation code paths14:42
ayoungmorgan, so the data we want is all in the extra14:42
morgandon't try and use the same table in this case.14:42
*** nbloom2 has left #openstack-keystone14:42
*** mylu has quit IRC14:42
morganayoung: yes. it is in "extra" [it was convenient place to store it, not beause users can wedge things into extra here"14:42
*** pnavarro has quit IRC14:44
edtubillHi, I was wondering if it was possible to use horizon with keystone to keystone federation (keystone Idp with websso)? I've been having troubles trying to make the redirection work (Not sure how the redirection works).14:45
morganayoung: so the question is... do you want to take the short path to unified token validation? or keep on the path we've been on which is retrofit things into the structures we have (and it's been very slow)14:45
rodrigodsedtubill, it is not :(14:45
rodrigodsstevemar, ^ right?14:45
morganrodrigods: it might be... but it probably requires custom code.14:46
rodrigodsmorgan, sure14:46
rodrigodsi mean, with the upstream merged code14:46
ayoungmorgan, ok, no new token provider14:46
morganayoung: i think wedging the payload-formed-uuid into that table is a recipe for disaster.14:46
ayoungits going to cause more pain14:46
ayoungand we can work with what we have14:46
morganayoung: i disagree 100%14:46
ayoungkeep the path the same for now, with the exception of this:14:47
ayoungwhen fetching token data from the peristance driver, only return the fernet  payload....14:47
ayoungok, we can do THAT as a new provider14:47
ayoungthe rest stays the same14:47
morgani aslo think that is a bad idea14:47
ayoungwe persist the same data that we do now, so that someone can switch back and forth14:47
rodrigodsedtubill, we need to make possible for horizon to talk with keytoneauth's plugin14:47
*** e0ne has quit IRC14:47
ayoungI don't want to dump the token table.  People are annoyed on upgrades already14:48
edtubillrodrigods, morgan: thx , and I was also wondering what this line does for configuring apache2: 'WSGIScriptAliasMatch ^(/v3/OS-FEDERATION/identity_providers/.*?/protocols/.*?/auth)$ /var/www/keystone/main/$1' Apache seems to break if I have that line. Do I need this line for Mitaka?14:48
rodrigodsedtubill, doug-fish was one of the ppl in this front14:48
morganayoung: so lets hold until the summit14:48
ayoungedtubill, that is for federation, and the fact that it is a wildcard looks wrong14:48
morganayoung: i don't think we're going to come to an agreement here on irc.14:48
ayoungmorgan, alternatively, we have a fallback from the new UUID provider to the old14:49
morganayoung: i think we're in for a world of hurt if we try and wedge things into the current model14:49
edtubillrodrigods: okay. I guess there's future work for horizon and keystonauth.14:49
morganasking people to re-authenticate if you need to switch providers is *not* awful14:49
*** josecastroleon has quit IRC14:49
morganand i'm willing to conceed a migrate option from keystone-manage14:49
morganfrom old uuid to new14:50
rodrigodsedtubill, https://review.openstack.org/#/c/159910/14:50
patchbotrodrigods: patch 159910 - django_openstack_auth - K2K federation14:50
morganmy reasoning for wanting to pivot to a new table really is so we don't need to pull the whole auth response.14:50
morganand we don't need magic to understand the differences14:50
morganand it means when the deprecation cycle goes through we drop code14:51
morganwith a large cut (and a drop of the old table)14:51
*** mylu has joined #openstack-keystone14:52
ayoungmorgan, all this is to migrate to a new token provider that we don't even want14:53
morganayoung: uuid is not going to die soon.14:53
morganayoung: it's a lot less code to pivot, no retrofitting. and it gets us what you asked for a lot sooner.14:54
morganand isolates code paths.14:54
edtubillrodrigods: Iooks like an old patch. So is that the k2k code for horizon and the k2k auth plugin?14:54
morgani really am looking at it from a "shortest path to where we want to be in Newton" standpoint14:55
morganand with the easiest to drop old stuff - while not just breaking everything.14:55
*** phalmos has joined #openstack-keystone14:55
ayoungmorgan, I hear ya....just the pain of having uuids in two tables14:56
ayoungmigrating really is going to take a long time.14:56
morganayoung: so that is why it's a keystone-manage migrate_tokens14:56
morganand is optional14:56
ayoungI'm prone to say "try the new UUID, and if that fails, try the old" as a transition14:56
morganor something similar14:56
ayoungall writes go to the new table.14:57
ayoungThe token flush needs to flush both tables14:57
morganayoung: and the fallback is worth 1 day of validates? or 2?14:57
morganthat is a lot of code for a very small window14:57
*** timcline has joined #openstack-keystone14:58
morganif someone is swapping to fernet - people have to reauth14:58
ayoungI hear ya.  I don't like it either14:58
morganif someone is swapping from PKI -> uuid, they need to reauth14:58
ayoungpki is still persisted14:58
morganyes. in many cases.14:58
ayoungthat was just a flip of the switch...stop signing14:58
morganbecause bugs14:58
morgannot because bad architecture14:58
ayoungbad architect14:59
morganbut i'm willing to say swapping to the legacy-uuid is like changing to/from fernet14:59
morganand i'm willing to conceed a migrate from legacy->new if someone wants to14:59
morganbut my guess is they'll tell people "dude, reauth"14:59
mylurodrigods: I figured it out, I need to pass the shibboleth session cookie with the request afer 30215:00
ayoungso...how oabout this15:00
*** jaosorior has quit IRC15:00
ayoungfor a first proof-of-concept, lets write a driver that just extends the existing token drivers return data15:00
ayoungdoes the fernet values only15:00
ayoungbut pulls it from the existing data15:00
ayoungsee how bad that is15:00
ayoungthe rest of the token provider stays the same15:00
morganayoung: i'll review it, but if you're going to ask me to write it, i'm going further ;)15:01
ayoungmorgan, I think I can write that fairly quickly15:01
morganayoung: because i'm going to simply subclass fernet and ignore all the other code.15:01
ayoungmorgan, hmmm...I don'tthink that will be easier15:02
morgani really don't think you understand how much work it's going to be to figure out what to pull out of the json blob15:02
morganayoung: because remember fernet has different formats per type15:02
morganis this a trust token?15:02
morganis this scoped?15:02
*** rderose has joined #openstack-keystone15:02
ayoungmorgan, so Fernet does signing.  Thatis the part we need to bypass.15:02
morganthe payload is different15:02
morganthat is msgpacked15:03
morganand differs on the types of tokens.15:03
ayoungso we are going to keep the msgpacked part...that can't be in the token ID though15:03
ayoungnot random enough15:03
morgani was going to store the msgpacked part in the db15:03
morganand still use uuid.uuid4().hex for the id15:03
morganin the case15:03
morgani really really was serious about the difference being .decrypt() or .get_from_db(id)15:04
morganand 100% of the code is the same otherwise.15:04
ayoungok, so we'll have legacy-uuid, msgpack-uuid, fernet15:05
morganpretty much.15:05
ayoungso, how does that makethings better?15:05
ayoungI hear you that msgpack is a better UUID format15:05
morganlegacy-uuid is deleted in O15:05
ayoungbut it means that we still have the legacy, and thus all theproblems are still there15:06
morganmsgpack-uuid uses 100% the same code path as fernet15:06
morganwhich means we're testing the code paths equally15:06
morganit gets us to "fernet is being tested" except the cryptography libaray bits15:06
ayoungI think I still want to redo legacy uuid15:06
morganright now.15:06
ayounglet me think about15:06
ayounghow that falls out on the pain balance15:07
morgani think the pain is a reauth.15:07
*** mylu has quit IRC15:07
morganwhich is a low amount of pain tbh15:07
morganeverything out there has to understand how to reuath anyway15:07
ayoungmorgan, it means long runing tasks fail15:07
morganthey fail when you take keystone down15:08
morganfor upgrade anyway15:08
*** stingaci has joined #openstack-keystone15:08
morganand anyone who says we'll be on 100% rolling upgrades in N is (in my book) crazy15:08
morganwe'll be closer15:08
dstanekok, so it looks like something drop it's dep on testresources and now keystone tests fail15:08
*** josecastroleon has joined #openstack-keystone15:08
morgandstanek: oh wonderful.15:09
*** trown is now known as trown|afk15:09
morgandstanek: i am so tired of python requirement resolution :(.15:09
*** mylu has joined #openstack-keystone15:10
morganayoung: stew on this convo for a bit. lets circle up at the summit15:10
morgani think we can still hit N1 if we discuss there15:10
dstanekmorgan: yeah, going to submit a few patches to fix15:10
ayoungmorgan, will do.  One nice thing about this approach is we will tell people "since you are going to have to reauth anyway, you might as well switch to Fernet"15:11
bknudsonwhat's failing? py27?15:11
morganayoung: and the only reason i'm not rage-coding (ok not rage-coding) my proposal right now, is because i have to get my new internet installed today and need breakfast15:11
morganayoung: ;)15:11
bknudsonI just ran it 1/2 hour ago15:11
ayoungbknudson, in a clean venv?15:11
bknudsonI rm'ed .tox because I figured deps had changed due to eventlet removal15:12
morganbknudson: sufficient for new venv then15:12
ayoungdstanek, what commit is the first broken?15:12
morganayoung: woot 1GB internet/fiber at home!15:12
ayoungmorgan, nice15:12
ayoungmorgan, I'm at Dunkin Donuts on wireless15:12
morganmy poor wifi network will be the bottleneck now15:13
ayoungnot quite as snappy15:13
*** c_soukup has joined #openstack-keystone15:13
stevemarayoung: maybe DD upgraded, anything can happen15:13
dstanekayoung: it's broken on master for me an apparently on stable branches as well15:14
*** csoukup has quit IRC15:16
ayoungstevemar, DD Wifi actually is pretty good, and fewer people on it here than at the cowork space15:17
*** stingaci has quit IRC15:18
ayoungmorgan, if I can get UUID working  without the token dump to start, we can do the full msgpack approach without the pressure.  Its not either-or.  My approach is a change in the validation logic which will need to be there anyway.15:20
ayoungI think I'll give it a stab later on today.15:20
bknudsondstanek: what's the failure?15:20
ayoungwe can always chose not to use it.15:20
*** phalmos has quit IRC15:20
ayoungmoving locations...back in a bit15:20
*** ayoung has quit IRC15:20
dstanekbknudson: testresources isn't installed. i'm creating a bug for it so that i can track my patch against it.15:21
bknudsonit's working fine for me.15:21
bknudsonmaybe it's getting a package from local cache or something.15:21
dstanekbknudson: really? on master i just 'tox -re py27' and i don't get it installed15:21
bknudsonpip freeze shows testresources==1.0.015:22
bknudsondstanek: yes, I just ran it and no errors15:22
dstanekit also looks like there is an aemail about the stable branch on -dev15:22
dstanekbknudson: let me clear all the package caches and try again15:23
arunkantstevemar, gordc: Can you review audit middleware change ..https://review.openstack.org/#/c/279828/ ..it has been pending for a while.15:26
patchbotarunkant: patch 279828 - keystonemiddleware - Adding audit middleware specific notification driv...15:26
*** phalmos has joined #openstack-keystone15:28
openstackgerritDavid Stanek proposed openstack/keystone: Explicitly require testresources for tests  https://review.openstack.org/30787815:29
dstanekbknudson: i still get the issue and ^ is that fix that works15:29
rodrigodsmylu, awesome!15:29
bknudsonif we use testresources directly then we should include that.15:30
bknudsondstanek: can you post your pip freeze and I'll compare with mine15:30
dstanekbknudson: we do in a way because we use oslo.db's test base class and it uses testresources15:30
bknudsonthen oslo.db should include testreources?15:31
dstanekbknudson: oslo.db only requires testresources for it's tests and now when it is installed15:31
dstanekis a gray area because oslo.db doesn't need it to work. the project using oslo.db is what needs it for only tests15:31
bknudsonyou need oslo.db[fixtures]15:32
dstanekwill that install the right things?15:32
bknudsonwhich we've got in keystone test-requirements.txt15:32
bknudsondstanek: try .tox/py27/bin/pip install oslo.db[fixtures[15:32
dstaneki have that in my test-r.txt as well15:33
openstackgerritChristopher J Schaefer proposed openstack/python-keystoneclient: Removing bandit.yaml in favor of defaults  https://review.openstack.org/29459715:33
openstackgerritChristopher J Schaefer proposed openstack/python-keystoneclient: Removing bandit.yaml in favor of defaults  https://review.openstack.org/29459715:35
dstanekbknudson: here is the output of pip freeze https://gist.github.com/anonymous/e0a6414197dc24a4901a4c1b8cefe31815:37
*** josecastroleon has quit IRC15:38
dstanekbknudson: actually if i explicity pip install oslo.db[fixtures] it works15:38
bknudsonsomething strange is going on.15:38
bknudsondstanek: is that pip freeze for you whole system or just the .tox/py27?15:39
dstanekbknudson: oh, woops that was the wrong venv...jas15:40
dstanekbknudson: https://review.openstack.org/#/c/307858/2/global-requirements.txt15:40
patchbotdstanek: patch 307858 - requirements (stable/kilo) - Cap testresources<2.0.015:40
dstanekmaybe related to my issue...15:40
*** stingaci has joined #openstack-keystone15:40
bknudsonmy venv gets testresources==1.0.015:40
dstanekmy gets 2.0.0 when i run manually. nothing when i don't. you must be using a cache of some sort since testresouces release 2.0.0 the other day15:41
bknudsonseems to work fine with testresources==2.0.015:41
dstanekbknudson: it's installing that automatically for you?15:42
bknudsondstanek: no, I did .tox/py27/bin/pip install -U testresources15:42
dstanekbknudson: yeah, it works fine. the issue for me is that it's not installed15:42
bknudsondstanek: post your pip freeze15:42
bknudsonand I agree it's strange I'm not getting testresources215:43
dstanekbknudson: are you using a custome pypi index or local cache?15:44
bknudsondstanek: here's the diff: http://paste.openstack.org/show/494672/15:44
bknudsonso it's just the oslo.db stuff.15:45
*** links has joined #openstack-keystone15:45
bknudsonI think pip caches locally by default.15:45
bknudsonI'm not doing anything special with pypi indexes or caching.15:46
bknudsonyou're the one with wheel==0.24.015:47
dstanekyeah, i have no idea where that came from15:48
dstanekok, so fun fact. i just updated tox and pip in the system itself and i get testresources 1.0.015:49
bknudsonwhat versions were you running?15:49
dstaneknot that old...jas15:50
*** browne has joined #openstack-keystone15:50
dstanekbknudson: http://paste.openstack.org/show/494674/15:51
bknudsonthose were old.15:52
bknudsoncattle not pets.15:52
*** josecastroleon has joined #openstack-keystone15:52
*** daemontool has quit IRC15:52
bknudsonI'm running the same versions. Not sure when this changed.15:52
*** daemontool has joined #openstack-keystone15:53
bknudsonwe have minversion = 1.6 in tox.ini, maybe that should be upped?15:53
*** gyee has joined #openstack-keystone15:54
*** ChanServ sets mode: +v gyee15:54
*** lhcheng has joined #openstack-keystone15:55
*** ChanServ sets mode: +v lhcheng15:55
bknudson./neutron/tox.ini:minversion = 2.3.115:55
bknudsonthe rest are 1.6, 1.8, 1.415:55
bknudson./tempest/tox.ini:minversion = 2.3.115:55
bknudsonthat's probably when I upgraded.15:55
*** timcline has quit IRC15:56
*** links has quit IRC15:57
*** gokrokve has joined #openstack-keystone15:58
*** doug-fish has quit IRC16:00
*** Guest73397 has quit IRC16:00
*** doug-fish has joined #openstack-keystone16:00
*** zzzeek has quit IRC16:02
*** zzzeek has joined #openstack-keystone16:03
*** doug-fish has quit IRC16:09
*** dan_nguyen has joined #openstack-keystone16:10
*** ayoung has joined #openstack-keystone16:11
*** ChanServ sets mode: +v ayoung16:11
*** manjeets has joined #openstack-keystone16:14
*** sdake has joined #openstack-keystone16:14
manjeetsis there any config type option for keystone where you can say use v2 instead of v3 ?16:14
dstanekbknudson: i just downgraded tox and pip, but it still works - was hoping to nail down what was the actual cause16:14
manjeetsi was using devstack but seems like v3 is forced now16:15
stevemardstanek: i'm going to head out now to a court house (gotta defer jury duty), on the odd chance that i'm late, can you run the meeting :)16:16
dstanekstevemar: sure16:16
stevemarit shouldn't take long, but just in case, i don't want you guys waiting on me to start16:16
*** samueldmq has quit IRC16:17
*** anteaya has quit IRC16:20
*** josecastroleon has quit IRC16:22
*** pauloewerton has joined #openstack-keystone16:23
*** phalmos has quit IRC16:23
*** rderose has quit IRC16:24
*** rderose has joined #openstack-keystone16:25
*** sdake_ has joined #openstack-keystone16:25
*** doug-fish has joined #openstack-keystone16:27
*** mylu has quit IRC16:27
*** mylu has joined #openstack-keystone16:27
*** TxGVNN has quit IRC16:29
*** phalmos has joined #openstack-keystone16:29
*** sdake has quit IRC16:29
*** mylu has quit IRC16:33
*** gyee has quit IRC16:34
*** mhickey has quit IRC16:37
*** jistr has quit IRC16:39
*** stingaci has quit IRC16:44
*** fawadkhaliq has joined #openstack-keystone16:45
openstackgerritMerged openstack/keystoneauth: Fixing D301 docstring.  https://review.openstack.org/30758716:46
*** pumarani__ has joined #openstack-keystone16:47
*** pushkaru has quit IRC16:50
*** henrynash has quit IRC16:51
*** timcline has joined #openstack-keystone16:54
rodrigodsbknudson, dstanek, stevemar https://review.openstack.org/#/c/298696/16:55
patchbotrodrigods: patch 298696 - openstack-infra/project-config - Enable non-voting keystone tempest plugin tests16:55
rodrigodsshould be running today! :)16:55
*** pumarani__ has quit IRC16:58
*** samueldmq has joined #openstack-keystone16:59
*** ChanServ sets mode: +v samueldmq16:59
openstackgerritayoung proposed openstack/keystone: [WIP]Make fernet default token provider  https://review.openstack.org/25865017:00
*** browne has quit IRC17:02
*** rcernin has quit IRC17:04
openstackgerritAlexander Makarov proposed openstack/keystone: Add set_config_defaults() call to tests  https://review.openstack.org/30467417:06
*** stingaci has joined #openstack-keystone17:07
*** tqtran has joined #openstack-keystone17:08
*** josecastroleon has joined #openstack-keystone17:10
*** EinstCrazy has quit IRC17:14
openstackgerritMerged openstack/keystoneauth: Fixing D204, D205, D208, and D211 pep8  https://review.openstack.org/30759717:15
*** slberger1 has joined #openstack-keystone17:15
*** slberger has quit IRC17:16
openstackgerritwerner mendizabal proposed openstack/keystone-specs: Credential Encryption  https://review.openstack.org/28495017:33
*** gokrokve has quit IRC17:37
openstackgerritayoung proposed openstack/keystone: Make all fixture project_ids into uuids  https://review.openstack.org/30668117:37
*** gokrokve has joined #openstack-keystone17:37
*** gyee has joined #openstack-keystone17:39
*** ChanServ sets mode: +v gyee17:39
*** nkinder_ has quit IRC17:39
openstackgerritayoung proposed openstack/keystone: Make fernet support trust auth against v2.0  https://review.openstack.org/27869317:39
*** josecastroleon has quit IRC17:40
openstackgerritAlexander Makarov proposed openstack/keystone: Unified delegation assignment driver  https://review.openstack.org/29131817:42
*** stingaci has quit IRC17:43
*** dave-mccowan has quit IRC17:45
*** stingaci has joined #openstack-keystone17:47
*** browne has joined #openstack-keystone17:48
htrutaayoung: are you around? just a quick doubt. How did access control worked in keystone before policies? Reading the release notes from Essex it looks like it had two apis, one for admin and one for user, right?17:49
ayounghtruta, guh....I really don't know17:50
ayoungbefore policies?17:50
ayoungI think it was there from the beginning...let me see17:50
htrutaayoung: looks like policies were implemented in grizzly17:51
ayoungpolicy was there in Diablo/Essex termie's branch https://github.com/termie/keystonelight/tree/master/keystone/policy17:51
morganso, policy was very very basic17:53
morganv3 has always had policy (ish)17:53
ayoungah...but was not a JSON file17:53
morganv2 was hardcoded mostly17:53
htrutaayoung: hm... so, the policies existed, but hardcoded and only admin x user17:53
ayounghtruta, yep17:53
morgan(how do you think we inherited that for so long;)17:53
htrutamorgan: nice!17:53
htruta(or not nice, depending on the view point)17:53
morganayoung: as someone who's worked in security and security-adjacent fields for a while... https://news.bitcoin.com/looting-fox-sabotage-shapeshift/17:54
morganayoung: i read this and it's interesting to see folks get bit (again) and re-learn the infosec lessons in this current wave of startup-land17:55
*** shaleh has joined #openstack-keystone17:55
htrutamorgan, ayoung: AFAIK first version of keystone api was v2.0, right? The v1, which received only headers as args is from the time nova was still responsible for auth17:55
htrutacorrect me if I'm wrong17:55
ayounghtruta, right17:55
morganhtruta: so V1 was nova, v2 was keystone-... lite?...17:55
ayoungthat was internal to RAX, pre me17:55
morganand v3 was "oh god what have we done... no we need to fix that"17:56
rodrigodswhere are the dinosaurs?17:56
ayoungI joined the effort in Dec  of 2012?17:56
ayoungI've spent my forties doing openstack17:56
htrutaayoung: you worked in Essex. That's why I asked you: https://launchpad.net/keystone/essex/2012.117:57
ayounghtruta, oh yeah...I am aware.  Just repressed memories17:57
morgani started contributing in Essex (nova)17:57
morganand i think i landed patches in keystone in grizzly17:57
htrutaayoung, morgan: thanks guys!17:57
ayoungI was made core by Joe Heck and dolphm because they need a thrid person to help code review.   And termie had disapparated17:57
raildothe keystone meeting will be to tell the Keystone history :)17:57
morganoh hey it's meeting time17:58
ayoungI took the old Nova based LDAP code and corrupted termie's KSL port17:58
stevemarmorgan: soon :)17:58
rodrigodsayoung, third core?17:58
* morgan just realized it was tuesday17:58
ayoungrodrigods, yep17:58
morganrodrigods: 4th.. technically.17:58
morganrodrigods: cause #termie17:58
rodrigodsmorgan, hmm true17:58
morganbut he was MIA17:58
ayoungmorgan, more than that, there were others, just not active17:58
ayoungI was active17:58
morganayoung: so when are we doing keystone v4? :P17:59
raildomorgan: lol17:59
ayoungmorgan, so I thikn we can get to tokenless without a v417:59
morganayoung: so do i17:59
rodrigodswe can17:59
ayoungand with that17:59
gyeetokenless ftw!17:59
*** henrynash has joined #openstack-keystone18:00
*** ChanServ sets mode: +v henrynash18:00
morganoh man. the opensrtack css died on specs.openstack.org18:00
rodrigodsgyee, now we know how to summon you18:00
gyeeuse certs18:00
morgangyee: shhh18:00
*** doug-fis_ has joined #openstack-keystone18:00
morgangyee: also no. OAuth18:00
gyeeI had a demo of that in the last meetup18:00
openstackgerritBoris Bobrov proposed openstack/keystone-specs: Functional testing setup  https://review.openstack.org/30737118:00
openstackgerritBoris Bobrov proposed openstack/keystone-specs: Federation testing setup  https://review.openstack.org/30796018:00
morgani think i want to see if we can get https://specs.openstack.org/openstack/keystone-specs/specs/keystone/backlog/decouple-auth-from-api-version.html in newton18:01
*** doug-fi__ has joined #openstack-keystone18:01
*** mtreinish has quit IRC18:03
*** doug-fish has quit IRC18:04
*** doug-fis_ has quit IRC18:04
*** dave-mccowan has joined #openstack-keystone18:05
*** mtreinish has joined #openstack-keystone18:08
*** timcline has quit IRC18:08
*** timcline has joined #openstack-keystone18:09
openstackgerritMerged openstack/keystone: Updating sample configuration file  https://review.openstack.org/30760618:11
*** henrynash has quit IRC18:17
*** edtubill has quit IRC18:19
*** henrynash has joined #openstack-keystone18:23
*** ChanServ sets mode: +v henrynash18:23
stevemarshaleh: i liked your response to OSC's slowness (in regards to rust and go)18:29
morganstevemar, shaleh: the question comes -- how do you force a refresh of the stevedore cache? since that is heavy inspection - on new install?18:31
shalehmorgan: --clear-cache18:35
shalehit will live in ~/ or a location specificed18:35
*** doug-fi__ has quit IRC18:36
shalehthe modules only change when a new install is done18:36
shalehwhy pay for dynamism when we almost never need itr18:36
shalehmonty's solution of pure GET/POST is OK if we have to do it. But we all put effort into our *client libs for a reason.18:37
shalehmorgan: I do not know if stevedore could be replaced or optimized in this way18:37
shalehmorgan: but it seems like an obvious place to experiment18:37
morganshaleh: uhmmmmmmmmmmm i am not sure if that is a good plan --clear-cache.. will need to think about it18:38
gyeeso stevedore doesn't cache stuff?18:38
shalehgyee: not that I can see18:38
shalehgo look in all of the eggs that say "openstack..."18:38
* dhellmann might be happy to see a patch adding that feature18:38
gyeeI would think it should, as its based on *registration*18:39
gyeeregister once18:39
shalehgyee: someone (maybe me) needs to dig into why the module loading is slow18:39
gyeedlopen? :-)18:39
gyeeunder the hood I mean18:39
shalehthe fellows hack of having a OSC service for devstack is still a good one regardless of what we is done. Why pay for any startup time when we know we are just going to pelt it with requests18:40
*** dave-mccowan has quit IRC18:40
*** sdake_ has quit IRC18:41
*** josecastroleon has joined #openstack-keystone18:41
shalehstevemar: I like Rust over C/C++ and Go. But still, rewriting means bugs, optimizing new issues, etc.18:41
shalehplus getting Rust (which is a moving target) stable enough to put on a system and not touch it for 2 years18:41
shalehGo is a little better in that regards18:42
*** trown|afk is now known as trown18:43
morganroxanaghe: i want to put you in touch with cburgess, re: ldap things18:43
morganroxanaghe: since we're re-working the ldap driver. some improvements once we have parity would be good (re filtering, etc)18:43
*** itlinux has quit IRC18:45
roxanaghemorgan, that would be great!18:46
roxanaghemorgan, this is my latest version of mockSync: https://github.com/roxanagherle/ldap3/blob/master/ldap3/strategy/mockSync.py I wrote a message to the ldap3 owner, since I wanted to get his opinion if he wants something like that in the ldap3 repo18:47
*** sdake has joined #openstack-keystone18:48
morganroxanaghe: nice18:48
roxanaghemorgan, filtering on an ldap query is a beast18:50
*** KarthikB has joined #openstack-keystone18:50
roxanaghemorgan, so I just took whatever they did in mockldap library for python-ldap for now18:50
morganroxanaghe: yeah it is. cburgess is running into issues so i figure we can work on improving it18:51
roxanaghemorgan, ok let me know how I can help18:53
*** henrynash has quit IRC18:59
bknudsonayoung: when you push a new version the -1 will go away19:00
samueldmqsee ya in austin19:00
lbragstaddolphm do you want to go through and add the edits from the talk?19:00
*** fawadkhaliq has quit IRC19:00
dolphmlbragstad: i've been doing some already19:00
dolphmlbragstad: i added the 'what is a token' section and 'what is a fernet key'19:01
lbragstaddolphm sweet - reviewing19:01
erhudyis there an explanation of the PKI token format anywhere? i'm digging into something with the v2.0 API and when i base64-decode a token it looks like mostly JSON but with some byte noise surrounding it19:03
gyeeerhudy, see PKCS #719:04
*** doug-fis_ has joined #openstack-keystone19:04
ayounghttps://review.openstack.org/#/c/195780/ bknudson19:05
patchbotayoung: patch 195780 - openstack-dev/devstack - Switch fernet to be the default token provider19:05
*** sheel has quit IRC19:05
*** sigmavirus24 is now known as sigmavirus24_awa19:06
rodrigodsbknudson, samueldmq, rderose https://bugs.launchpad.net/keystone/+bug/1571878 makes sense?19:07
openstackLaunchpad bug 1571878 in OpenStack Identity (keystone) "Add protocol to identity provider using nonexistent mapping" [Undecided,New] - Assigned to Ron De Rose (ronald-de-rose)19:07
rodrigodsthe last comment19:07
*** neophy has joined #openstack-keystone19:08
*** doug-fis_ has quit IRC19:09
*** KarthikB has quit IRC19:12
*** KarthikB has joined #openstack-keystone19:12
*** josecastroleon has quit IRC19:12
*** doug-fish has joined #openstack-keystone19:14
morganbknudson: positional 1.1.0 with wrapt is now released19:15
samueldmqrodrigods: I agree, however not sure about deprecating a workflow19:17
samueldmqrodrigods: not sure how that will map to deprecating APIs, but we'll see19:17
*** KarthikB has quit IRC19:18
*** real56 has quit IRC19:21
morganayoung: ok i think i'm gonna hack on token things19:25
morganayoung: this is bothing me and i want to see how it goes.19:25
morganrodrigods: i'll also look at mocksync in a few minutes.19:25
*** mylu has joined #openstack-keystone19:25
morganrodrigods: not rodrigods roxanaghe19:25
ayoungmorgan, go for it.  But I hate the idea of dumping the token table.  It is the kind of pain I want to no longer propagate.19:26
morganayoung: i think it'll be easier to transition this way tbh19:26
ayoungmorgan, also...I don't like the idea of making a table with the msgpack in it for differnt reasons19:26
morganayoung: i expect we'll discuss @ the summit :)19:26
ayoungI am ok with a table that has the columns in it in normal formal19:26
*** samueldmq has quit IRC19:26
ayoungnormal form19:27
morganayoung: i think this is a case where normal form is wrong .19:27
ayoungthe msgpack is unnecessary overhead19:27
morganunless we want divergent code poaths19:27
morganlike i said, my goal is identical validation(s).19:27
morganthat expand with forms of tokens in the same way19:27
ayoungmorgan, the path can join with the data returned post the msgpack processing19:27
morganayoung: except each column requires a migration if we add19:28
morganwhere fernet uses a formatter19:28
morganto build the payload19:28
ayoungyeah, well that is motivcation not to add more columns19:28
morganso you have divergent code paths19:28
morganagain, i am thinking the token payload should be the same.19:29
morganso we have one mechanism for token validation19:29
morganrather than... a few things that kindof work sortof the same19:29
morganif we're good at testing19:29
ayoungmorgan, "the same" meaning19:29
morgan(which is what we have now)19:29
morganmeaning, the *same* code19:29
morganusing the fernet formatter(s)19:29
ayoungmorgan, um...payload meaning what?19:29
morganjust not the encryption19:30
ayoungfor the UUID token?  We can't19:30
morganpayload is scope, trust, etc19:30
ayoungit has to be a uuidgen -r19:30
morganthe only thing the new table would encode is "uuid" index19:30
ayoungso we persist what you are aclalling payload inthe db,19:30
morgancreation time (fernet spec)19:30
morganand payload (the rest)19:30
morganthe fernet formatter is tied to the payload19:30
morganFERNET(HMAC(Create_time, AES(PAYLOAD)))19:31
morganis what we end up doing19:31
*** gyee has quit IRC19:31
morgani am proposing UUID is = UUID-index, Create_time, PAYLOAD19:31
morganstore the data the same way. validate the same way19:31
*** KarthikB has joined #openstack-keystone19:31
lbragstaddolphm presentation looks good to me19:32
morganthe difference is if the payload is AES w/ HMAC or .query(uuid-index)19:32
lbragstaddolphm thanks for fixing up those slides19:32
ayoungmorgan, meh....I have a feeling this is going to come back to bite me, but go for it19:33
ayoungyou care far more than I do19:33
morganayoung: i'm just trying to get us to "fernet is tested as well as other provider(s)"19:33
morganwhich means we don't have tests that diverge because the code paths are wildly different19:33
ayoungmorgan, I hear that, but that requires changes of the tests, not a new uuid provider19:34
morganexcept we suck at having divergent paths tested equally19:34
ayoungmorgan, no, there were basic code paths in Fernet that were untested19:34
morganlook at even PKI and UUID19:34
ayoungPython34 was failing on tests that just were not run19:34
morganthey were almost the same and the tests still were bad.19:34
ayoungI don't want another UUID provider19:34
morganand if we need to maintain uuid, we should not have it validate in a wildly different way than fernet19:35
morganand i don't think we're getting rid of uuid anytime soon19:35
ayoungOK...go on and code.  We'll deal with the fallout19:35
morganwe'll see how it goes.19:36
morganit may be horrific19:36
morganbut it looks pretty darn doable19:36
* morgan is only thinking with a way for folks who may have subclassed current uuid to not be broken, otherwise i'd advocate just changing the current provider19:36
ayounglet me just state the things I hate about this approach for the record: 1.  Another provider.  2. Dumping the token table, 3.  The serialized form in the DB is completely not-readable.  Other thanthat...meh19:37
morgan1 and 2 are mitigatable if we don't mind changing the current provider out19:38
morgan3 - i *could* just use json. but prefer to keep it the same code as fernet.19:39
morganthe only issue with #2 is the migration is brutal.19:39
*** woodburn has joined #openstack-keystone19:39
morganso *shrug*.19:39
ayoungmorgan, or if you do 3, you could avoid 219:39
morgannot really19:39
ayoungserialze to JSON you could keep old tokens19:40
morgannot really19:40
ayoungthat was my plan19:40
morganbecause we then need magic to know what the hell the old token forms are19:40
morganwe're in exactly the same place as we are now...or worse19:40
shalehwhy is coverage not enforced?19:40
ayoungmorgan, not really,  it is the token response that gets serialized.  Same data as is in our API19:40
shalehthat would solve the missed paths issue19:40
shalehif coverage < SOME_VALUE: boom()19:41
morganayoung: that is the problem! we are serializing the whole bloody thing and that has a miriad of variations and issues19:41
ayoungshaleh, so certain tests assume that the token provider is irrelevant19:41
*** mtreinish has quit IRC19:41
morganshaleh: there is math that prevents that from working19:41
ayoungmorgan, we serialize only V319:41
morganayoung: wrong.19:41
shalehmorgan: oh?19:41
ayoungmorgan, we do now19:41
morganwe serialize v2 if it is requested19:41
ayoungand if we don't we can serialize just v319:41
*** josecastroleon has joined #openstack-keystone19:41
morgananyway. i am against holding the whole token body for many reasons19:42
morganit's silly19:42
ayoungpretty sure a v2 request is serialized as v3 and converted to and fro19:42
ayoungI agree.19:42
morganso, we should stop doing that19:42
ayoungBut then Keystone is silly19:42
*** mtreinish has joined #openstack-keystone19:42
morganayoung: this is a case of not being opinionated enough that we have multiple ways of doing something and we have potential to leak impl details19:42
ayoungbut seriosuly, the only part I really care about is the operator pain.19:43
morganto the end user19:43
morgani'm really trying to not end up there. and operator pain is minimal if you're providing an option of "reauth" or "migrate tokens" and they can pick19:43
ayoungmorgan, OK, so the Fernet by default is now passing py2  and py319:43
morganayoung: however i have to be a hard -1 on making fernet the default in keystone.config due to op overhead of setup and cluster issues19:44
morganayoung: unfortunately.19:44
ayoungmorgan, no, that is fine.  I think we can do this:19:44
ayoung1.  Make that test run19:44
ayoung2.  RE install the caching19:44
morganayoung: i don't want to block it :( I really don't.19:44
ayoung3.  make sure we are invalidating cache enough to get test to run again19:45
ayoungdrop the "Fernet is the default"19:45
ayoungthat was 4.19:45
morganshaleh: so, if you remove code - enough code, you can actually get coverage % to reduce without losing coverage19:45
ayoungand merge the patch19:45
morganshaleh: which is why we can't enforce.19:45
ayoungthen follow on patch that expands test coverage19:45
shalehmorgan: sounds fishy.19:46
morganshaleh: and it's VERY hard to know if code path X is still code path X or code path Y now (it moved?) to check if coverage was gained or lost19:46
morganshaleh: example:19:46
morganshaleh: i delete the eventlet code, and all tests associated to it19:46
morganassume that is 1% of the tests but only 3 lines of code [contrived]19:46
morganshaleh: we would have a net loss of coverage %19:47
*** raddaoui has quit IRC19:47
shalehmorgan: not sure it is as bad as you think19:47
morganor wait.. vice versa.19:47
morganshaleh: we can't enforce.19:47
shalehmorgan: so make it advisory and we monitor it.19:47
morganshaleh: burden of proof - prove to me enforcement WONT prevent patches from landing when we delete code that is to be removed. (math)19:48
shalehif you run this group of tests and there is not X coverage for this directory, flag it. not fail, but flag it.19:48
morganshaleh: and i'll support you, but last time we did this, we came up with "can't do it"19:48
morganit's already advisory ;)19:48
shalehmorgan: plenty of github and over projects enforce it :-)19:48
morganshaleh: and i think they basically either have 100% coverage or have a small contributor base / code base that just doesn't run into this19:49
shalehbut like I said, we could still have a "did the test cover X directory with Y percent? No, flag it." Not fail, flag.19:49
morganwe would have failed to remove a number of code paths already.19:49
morganwhat is "flagging it"?19:49
*** mtreinish has quit IRC19:49
morgandon't we already have a non-vote job?19:49
shalehmorgan: mark the the test result with "double check, it did not reach expected level"19:50
morganshaleh: uh so a non-vote task that is "fail"19:50
shalehif this is because of an influx of code, it means we need more tests19:50
morganshaleh: its not the influx that matters, its the deprecation/removal that does.19:50
shalehwith a little touching now and then I do not see why it could not help.19:51
morganfor the reason why we can't block and i guarantee a non-voting job will mostly get ignored19:51
shalehmorgan: but there would be data19:51
morganget us to 100% coverage and it's easy to make the math never fail19:52
shalehdata that can be used for better planning19:52
shaleh100% coverage is almost always silly19:52
*** neophy has quit IRC19:52
shalehit is about keeping it at a sane level19:52
morganor the reviewers can look at the coverage report19:52
morganand use the data we have19:52
shalehwhere is the coverage link when a test runs?19:52
openstackgerritNavid Pustchi proposed openstack/keystoneauth: Fix H405, D105, D200, and D203 PEP257  https://review.openstack.org/30801619:53
morganshaleh: to be clear, i am not saying we shouldn't have a check job. i just dislike a job that fails that is non-vote (expectedly)19:54
morganthat will never be converted to voting (unless we hit 100% coverage.. )19:54
morganiirc we had at least at one time a job that ran coverage19:54
shalehmorgan: it would be interesting to gather info on how often it would fail, what the level of coverage could be, etc.19:55
morganshaleh: also if you move code from path X to path Y, the same "deleting code" math could cause it to fail19:55
morganshaleh: because you could have a net reduction in test coverage in the old path19:55
shalehmorgan: COULD happen and DOES happen are two different things19:55
morganshaleh: it *will* fail at some point in any case where the coverage is not 100% and the failures are in most cases going to be erroneous19:56
shalehon any other project where I have seen coverage tests fail it has been because of an influx of untested code19:56
shalehthe number does not have to be 100. It can be 72 if we desire.19:56
shalehthe point is consistency.19:56
morganconsidering the amount of code we've been deleting, there has been at least 5 times in the last cycle19:56
morganwe would have had erroneous failures in code duce to reduction in perceived coverage19:57
shalehmorgan: out of how many commits?19:57
shalehNo reason the review cannot include some way to help it pass coverage once it is clear that is going to happen19:57
morganapprox 1000 commits19:58
morganerm 90019:58
morganhold on no19:58
shaleh5 - 20 real fails over even 500 commits is not bad19:58
shalehprobably twice or three times caught for real failures to maintain coverage19:59
morganless than 600 commits19:59
morgani'm saying i am against a blocking or failing test19:59
*** navidp has joined #openstack-keystone19:59
*** gagehugo has quit IRC19:59
morgani am not against a coverage job that presents data19:59
morganthe coverage job can even show the % change20:00
shalehI get that. I am only arguing for measure it to find out :-)20:00
morganjust do not make it fail20:00
morgani want a fail to indicate the coverage job is broken not a number has changed.20:00
morganif that makes sense20:00
shalehTime will tell on whether it ever makes sense.20:00
morganjust like the doc job20:01
shalehit might not. Or it might mean the occasional commit needs help passing the job.20:01
*** nkinder has joined #openstack-keystone20:01
morganit presents the data, but it shouldn't fail unless the doc rendering cannot occur20:01
morganso i'll support a coverage job, i'll support one that shows % change20:01
morgani wont support one that "Fails" based upon %20:02
shalehwhat I have seen is once a culture of "X percent coverage for all code" exists the breakage rarely happens20:02
*** aimeeU has quit IRC20:02
shalehI would be happy to see coverage numbers, voting or not20:02
shalehit was one of the things I liked about running the py3 tests20:02
morganshaleh: and like i said, happy to even have to show the delta in coverage20:02
morganjust *not* a test that fails because of delta of coverage20:03
* morgan has a meeting to run to20:03
shalehmorgan: that is the base any conversation should start from. Otherwise it is all conjecture and bikeshedding20:03
morgananyway. meeting time20:04
ayoungrodrigods, https://review.openstack.org/#/c/306681/2  seemy reply20:04
patchbotayoung: patch 306681 - keystone - Make all fixture project_ids into uuids20:04
*** sigmavirus24_awa is now known as sigmavirus2420:06
*** jaugustine has quit IRC20:08
*** mylu has quit IRC20:08
*** mylu has joined #openstack-keystone20:08
*** KarthikB has quit IRC20:11
*** josecastroleon has quit IRC20:11
*** KarthikB has joined #openstack-keystone20:11
*** henrynash has joined #openstack-keystone20:12
*** ChanServ sets mode: +v henrynash20:12
*** daemontool has quit IRC20:12
*** dave-mccowan has joined #openstack-keystone20:14
*** fawadkhaliq has joined #openstack-keystone20:14
*** dave-mcc_ has joined #openstack-keystone20:15
roxanaghemorgan, cool thanks let me know if you have any feedback20:15
*** KarthikB has quit IRC20:16
*** dave-mccowan has quit IRC20:18
*** mylu has quit IRC20:21
*** fawadkhaliq has quit IRC20:22
*** fawadkhaliq has joined #openstack-keystone20:22
*** KarthikB has joined #openstack-keystone20:24
*** doug-fish has quit IRC20:27
*** mylu has joined #openstack-keystone20:29
*** josecastroleon has joined #openstack-keystone20:30
*** KarthikB has quit IRC20:31
*** rderose has quit IRC20:32
*** KarthikB has joined #openstack-keystone20:32
*** tqtran has quit IRC20:32
*** mylu has quit IRC20:35
*** doug-fish has joined #openstack-keystone20:36
*** BigWillie has quit IRC20:36
*** doug-fish has quit IRC20:36
*** KarthikB has quit IRC20:36
*** doug-fish has joined #openstack-keystone20:36
*** iurygregory has quit IRC20:42
*** tristanC_ is now known as tristanC20:54
*** mylu has joined #openstack-keystone20:55
*** c_soukup has quit IRC20:57
bknudsonmorgan: thanks for the positional release! Now if we can get the debtcollector change merge / released we'll have usable docs for keystoneclient.20:58
*** josecastroleon has quit IRC21:00
*** mylu has quit IRC21:02
*** dims_ has joined #openstack-keystone21:02
*** dims has quit IRC21:02
morganbknudson: happy to help21:03
morganbknudson: i lost my gpg key :( so had to get a new one to relase that.21:03
*** mylu has joined #openstack-keystone21:04
bknudsonyikes. How do I know you're still you?21:04
*** pauloewerton has quit IRC21:05
openstackgerritSteve Martinelli proposed openstack/keystoneauth: Fix H405, D105, D200, and D203 PEP257  https://review.openstack.org/30801621:05
stevemarbknudson: considering he is morgan now and not notmorgan, can we really trust him?!21:06
*** gyee has joined #openstack-keystone21:07
*** ChanServ sets mode: +v gyee21:07
morganstevemar: you CANT21:07
*** dave-mcc_ has quit IRC21:08
*** mylu has quit IRC21:08
*** mylu has joined #openstack-keystone21:09
*** gordc has quit IRC21:09
*** trown is now known as trown|outtypewww21:12
*** BjoernT has quit IRC21:12
*** mewald has joined #openstack-keystone21:13
*** dave-mccowan has joined #openstack-keystone21:15
mewaldI am seeing this during my puppet runs for keystone: https://gist.github.com/mewald1/c7e33a1defb63511e302a0b8c64c5a8e any ideas?21:16
*** mewald has quit IRC21:17
morganroxanaghe: mocksync is looking good.21:19
morganroxanaghe: i like it a lot actually21:19
*** tqtran has joined #openstack-keystone21:19
morganthe direction looks solid, we just need to get real datasets (obv. and make it easy to do things with them)21:20
morganbut i think you're on the right path21:20
morganand it looks like nothing is wildly "private" interface wise...21:20
morganso it could be carried outside of ldap3 tree (mostly)21:21
roxanaghemorgan, cool - regarding carrying out of the ldap3 tree - it seems like you can't use an external object for the strategy, so we either have to get this in ldap3 tree or add the capability of an external strategy21:23
*** fawadkhaliq has quit IRC21:23
*** fawadkhaliq has joined #openstack-keystone21:24
roxanaghemorgan, that's why I'm trying to start a convo with the ldap3 guys21:24
morganroxanaghe: ah. lame.21:25
morganroxanaghe: but all sounds solid.21:25
morganand i 100% think it is the right direction to be going21:25
roxanaghemorgan, awesome!21:26
morganroxanaghe: do you have a backup strategy if you can't land it/support for it in ldap3?21:27
roxanaghemorgan, and thanks for the feedback21:27
morganmonkeypatch it in?21:27
shalehfork it on GitHub :-)21:27
bknudsonfork it and call it ldap421:28
shalehbknudson: nah, python-ldap3.121:28
roxanaghemorgan, I think it's cleaner this way, but the other way is to make ldap3 accept an external strategy and I'm hoping we should be able to convince the ldap3 guys on that21:28
*** fawadkhaliq has quit IRC21:28
morganroxanaghe: i mean, think of a backup plan if ldap3 accepts no code. because we will want this type of stuff anyway :)21:29
morganroxanaghe: don't need an answer today fwiw21:29
morganjust ponder it in case we need to figure it out21:29
shalehfrom looking at the code, external strategy provides plenty of good choices. upstream should be able to see that.21:29
roxanaghemorgan, right - I'm gonna think about that21:30
morganshaleh: i hope so21:30
*** sdake_ has joined #openstack-keystone21:30
morganshaleh: but contingencies on contingencies21:30
shalehmorgan: agreed21:30
*** mylu has quit IRC21:32
*** sdake has quit IRC21:33
*** josecastroleon has joined #openstack-keystone21:36
*** dave-mccowan has quit IRC21:37
*** mylu has joined #openstack-keystone21:39
*** mylu has quit IRC21:53
*** stingaci has quit IRC21:53
*** stingaci has joined #openstack-keystone21:53
*** ametts has quit IRC21:57
*** gokrokve has quit IRC21:59
*** sigmavirus24 is now known as sigmavirus24_awa22:02
*** phalmos has quit IRC22:04
*** stingaci has quit IRC22:06
*** stingaci has joined #openstack-keystone22:06
*** josecastroleon has quit IRC22:06
*** rderose has joined #openstack-keystone22:06
*** mylu has joined #openstack-keystone22:07
*** josecastroleon has joined #openstack-keystone22:14
*** fawadkhaliq has joined #openstack-keystone22:14
*** mtreinish has joined #openstack-keystone22:15
*** ayoung has quit IRC22:16
*** alex_xu has quit IRC22:16
*** timcline has quit IRC22:18
*** alex_xu has joined #openstack-keystone22:18
*** slberger1 has left #openstack-keystone22:20
*** ianw has quit IRC22:22
openstackgerritMerged openstack/keystonemiddleware: Updated from global requirements  https://review.openstack.org/30775422:23
*** ianw has joined #openstack-keystone22:24
*** sdake_ is now known as sake22:26
openstackgerritMerged openstack/keystoneauth: Updated from global requirements  https://review.openstack.org/30775322:37
*** ianw has quit IRC22:38
openstackgerritNavid Pustchi proposed openstack/keystone: Fix D400 PEP257  https://review.openstack.org/30806022:39
*** rderose has quit IRC22:40
*** josecastroleon has quit IRC22:44
*** KarthikB has joined #openstack-keystone22:49
*** tellesnobrega is now known as tellesnobrega_af22:52
*** KarthikB has quit IRC22:53
*** rderose has joined #openstack-keystone22:58
rderoserodrigods: sorry for the late response, yes it does make sense22:59
*** tellesnobrega_af is now known as tellesnobrega22:59
*** alex_xu has quit IRC23:05
*** josecastroleon has joined #openstack-keystone23:06
*** alex_xu has joined #openstack-keystone23:06
*** tqtran has quit IRC23:11
openstackgerritMorgan Fainberg proposed openstack/keystone: UUIDMsgPack Token Provider Added.  https://review.openstack.org/30806323:11
morganstevemar: ^ an example of using fernet code to handle UUID tokens.23:13
*** ianw_ has joined #openstack-keystone23:13
*** mylu has quit IRC23:14
*** david-lyle_ has joined #openstack-keystone23:14
*** david-lyle has quit IRC23:14
*** mylu has joined #openstack-keystone23:15
stevemarmorgan: you have my interest....23:17
morganstevemar: and really really small amounts of code considering23:17
morganstevemar: that could open the door to deprecating UUID provider itself and the entire persistence subsystem23:18
morganestimation in flush, and migrations, and tests, less than 300 lines of code23:18
morganand deprecation warnings.23:18
morganok ok... less than 40023:19
*** fawadkhaliq has quit IRC23:19
morganthere is some adjustments i'd want to make on the where "pack" is called23:19
morganinstead of it living on the formatter, move it to the provider.23:19
stevemarit would still need 2 cycles :)23:19
morgan1 ;)23:19
*** fawadkhaliq has joined #openstack-keystone23:19
morgannot API impacting23:19
openstackgerritRon De Rose proposed openstack/keystone: Move the assignment abstract base class out of core  https://review.openstack.org/29963523:20
morganstill most of the work is in those ~100 lines to make a UUID FernetPayload token23:20
*** navidp has quit IRC23:20
morganalso note i opted for uuid.uuid4().int for the token id, because... ints store better/index better in dbs :P23:21
openstackgerritMerged openstack/python-keystoneclient: Updated from global requirements  https://review.openstack.org/30777123:21
morgananyway.. i told ayoung it would be really small amounts of code.23:21
morganit is.23:21
*** sake is now known as sdake23:22
morganstevemar: and i think this is an easier way to pivot validation instead of trying to retrofit things into the current uuid validation code paths23:23
*** fawadkhaliq has quit IRC23:24
rodrigodsmorgan, will take a look in the code23:24
* rodrigods hopes to understand23:24
morganrodrigods: it is very straightforward code... which is to say fernet's architecture is way better than the old stuff.23:25
rodrigodsmorgan, the architecture i mean :)23:26
*** mou has joined #openstack-keystone23:28
mouHello, Openstack community23:29
moui have a tough question23:29
*** tqtran has joined #openstack-keystone23:29
mouwhat are proposed upgrade path for keystone from kilo to liberty release, without downtime?23:30
rodrigodsmorgan, really simple indeed :P23:30
rodrigodsgot the idea23:30
gyeemorgan, you have my curiousity23:30
rderosestevemar: I know we have a design session scheduled for "Shadowing LDAP users" under new features...  but how do I schedule a work session?23:30
*** stingaci_ has joined #openstack-keystone23:30
moui found very annoying feature which breaks my upgrade procedure23:30
*** stingaci has quit IRC23:30
morganmou: unfortunately, there will need to be downtime for the keystone upgrade. however, it should be minimal from the keystone side.I can't speak to any othe rproject23:31
mouin liberty padding in tokens was removed23:31
morganmou: but in short for keystone: turn off keystone, db_sync, turn on new keystone code.23:31
mouso i cant upgrade keystone servers one by one23:31
openstackgerritTin Lam proposed openstack/keystoneauth: Remove ClientException duplicate message property from BaseException  https://review.openstack.org/28575723:31
morganmou: keystone doesn't support no-downtime upgrades at this point - we are examining what we can to do get closer in newton (mitaka also does not support it), sorry23:32
morgangyee: take a look at the patch :)23:32
*** mylu has quit IRC23:32
*** mylu has joined #openstack-keystone23:33
moumorgan: my original plan was to shutdown one keystone, upgrade, and start, and move to another. i assume this plan due to no schema change (for my particular installation and used features).23:33
moumorgan: but changing token format is broke everything :((23:34
morganmou: running keystone on old schemas is never supported, the db schema is meant to be in lock-step23:34
morganmou: for now23:34
gyeewe taking about fernet? thought we accounted for both padding and no padding23:35
morganmou: which token format are you using?23:35
morgangyee: i thought we handled both cases too.23:35
moumorgan: but kilo can run on liberty schema (for my installation)23:35
morgangyee: well wait, OLD keystone can't read no-padding23:35
gyeemorgan, ahhh, right23:35
moumorgan: fernet23:35
morganmou: again, i apologize, but what you're doing is simply not supported nor tested23:35
*** josecastroleon has quit IRC23:35
mouAlso i wonder why does this patch was abandoned https://review.openstack.org/#/c/221799/ ?23:36
patchbotmou: patch 221799 - keystone (stable/kilo) - Remove padding from Fernet tokens (ABANDONED)23:36
morganthe liberty keystone can handle both padded and non-padded tokens. the kilo one only understands the padding23:36
*** mylu has quit IRC23:36
*** mylu has joined #openstack-keystone23:36
mouSo looks like i should build my kilo keystone with this patch, and continue23:37
morganmou: you are more than welcome to. I however HIGHLY recommend keeping your schema in lockstep23:37
morganand not running liberty on a kilo schema, no warranties or guarantees data wont be corrupted.23:37
morganmou: i do wish you good luck on it though :)23:38
moumorgan: thanks, but we tested kilo on liberty schema23:38
*** maestro2 has joined #openstack-keystone23:38
*** cburgess has quit IRC23:40
*** sudorandom has quit IRC23:40
moumorgan: sorry for disturbing you guys23:41
morganmou: you aren't disturbing us at all :)23:41
moujust spend 6 hours to figure out root cause of my problem and feeling lil exhauseted :( because i don't understand why this patch wasn't accepted :(23:41
morganmou: i just wish we had better news for you23:41
moumorgan: this patch is definitely good news for me :)))23:42
mouso i will go now and apply it :)23:42
*** josecastroleon has joined #openstack-keystone23:43
*** pleia2 has quit IRC23:43
*** sudorandom has joined #openstack-keystone23:43
*** mou has left #openstack-keystone23:43
*** pleia2 has joined #openstack-keystone23:43
morgandstanek: any experience with Pelican (static web site generator in python... like jekyll but... not)?23:45
*** cburgess has joined #openstack-keystone23:45
morgandstanek: i want to spin my personal website back up.23:45
*** dobson has quit IRC23:46
morgangyee: comments/thoughts welcome23:46
*** trey has quit IRC23:46
*** woodster_ has quit IRC23:48
*** dobson has joined #openstack-keystone23:49
*** trey has joined #openstack-keystone23:51
gyeemorgan, I see one benefit, which is no service interruption23:51
gyeebut why deprecate UUID?23:51
morganthe goal is to get our token validation to be 100% the same between the supported options23:51
morganand retrofitting old UUID paths with fernet paths is a lot of work23:52
morganeasier (much easier) to pivot to hooking into the hard work done to make fernet what it is23:52
morganand much cleaner23:52
gyeeonly difference is *always* require key management23:52
morganthis would be the "UUID" (in-db store) of tokens23:52
gyeeit may scare people23:52
morganand this can then exersize 100% of the code for fernet w/o keys23:53
morganno different than today's UUID for "Scary" wise23:53
gyeetoday's UUID does not require key management23:53
shalehmorgan: except for the Fernet is mildly complicatd for distributed keystones23:53
gyeekey management put us into a whole new different realm, in terms of security and compliance23:53
morgangyee: neither does UUIDMsgPack23:53
morganit doesn't use the fernet keys at all23:54
morganit stores the payload in the db23:54
gyeewe have to option to not do fernet?23:54
gyeesorry I haven't look at the code yet23:54
shalehjamielennox: I plan to play with your os-http tool. I have been using httpie. It almost looks like a baby version of httpie for OpenStack.23:54
morganthat is what this patch is :P23:54
shalehmorgan: it derives from Fernet though......23:54
morgangyee: go look https://review.openstack.org/#/c/308063/23:54
patchbotmorgan: patch 308063 - keystone - UUIDMsgPack Token Provider Added.23:54
gyeeoh, in that case, it's a win23:54
morganshaleh: it does derive all the validation work23:55
* gyee goes back to RTFC23:55
morganshaleh: it just changes instead of .fernet(payload) to .store_to_db()23:55
morganand .get_from_db() instead of .decrypt()23:55
*** browne has quit IRC23:55
*** tqtran has quit IRC23:56
*** tellesnobrega is now known as tellesnobrega_af23:56
morganshaleh: streamlining and reducing divergent code paths that do (**supposed to do**) exactly the same thing is good.23:56
gyeethat part I like23:56
gyeemost definitely23:56
gyeesecurity likes consistency and predictability23:57
*** david-lyle_ is now known as david-lyle23:57
* morgan might work on a security-ish project for my day job :P23:57
lbragstadrodrigods ping23:57
morganok... maybe i don't have a day job right now :P23:57
*** browne has joined #openstack-keystone23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!