Tuesday, 2016-08-23

*** david-lyle has joined #openstack-keystone00:00
jamielennoxmordred, stevemar, dtroyer_zz: i think i like the UX of passing app_name, app_version, that makes more sense than asking the user to pass a list for user_agent, and then shade and osc-lib can just add themselves to additional_user_agent00:00
dtroyer_zzjamielennox: (catching up a bit)  I don't have a strong need for a detailed user-agent, in my mind I would care about two things: knowing what is controlling the session layer (usually KSA now) and what is driving the REST routes and body. so in OSC that's KSA and the client lib/SDK.00:01
dtroyer_zzbut I can handle adding each layer if that's what folks want on the toher  end00:01
*** david-lyle_ has joined #openstack-keystone00:03
jamielennoxdtroyer_zz: for osc we would only need OSC and the client, it was when others start using osc_lib it'll be relevant00:04
jamielennoxbut whatever, the functionality can be there and if noone uses it it doesn't matter00:04
dtroyer_zzsure.  what I am unsure of right now is how osc-lib's version is useful in the user-agent, very little of it is responsible for what comes out in the REST.00:05
*** david-lyle has quit IRC00:05
ayoungIgorYozhikov, sounds like a regression, then.00:06
jamielennoxoh - then ok. we don't worry about osc-lib then00:06
jamielennoxi was mentally equating it to shade which is probably not true00:06
dtroyer_zznot so much today, and I'm not planning for it to grow command-specific stuff.  it's purpose is to encapsulate the stuff for making CLIs and plugins using a project client or SDK00:08
IgorYozhikovayoung, I'll try tomorrow with fresh installation and post the results00:09
*** david-lyle_ has quit IRC00:09
dtroyer_zzIgorYozhikov: one thing to check is which plugin is being selected.  run with —debug and look for a line early in the output 'Auth plugin XXXXXX selected'00:10
dtroyer_zzif its password, the problem is in our auto-detection of the —os-token and —os-url arguemnts overriding all else.00:10
*** edtubill has joined #openstack-keystone00:11
*** bigdogstl has joined #openstack-keystone00:15
dtroyer_zztry adding —os-auth-type token_endpoint to your command00:16
*** sdake has joined #openstack-keystone00:16
* dtroyer_zz mkaes note that those values should have dashes instead of underscores00:16
*** bigdogstl has quit IRC00:21
*** bigdogstl has joined #openstack-keystone00:21
jamielennoxhmm, probably - i'm not sure how we would fix that, i'm not sure that token-endpoint = is valid in setup.cfg00:21
dtroyer_zzwe convert all sorts of things back and forth, just need to do these too at the user-visible point00:23
*** ravelar has joined #openstack-keystone00:25
*** tonytan4ever has joined #openstack-keystone00:25
*** ravelar has quit IRC00:29
*** tonytan4ever has quit IRC00:31
IgorYozhikovdtroyer_zz, root@us16:~# openstack --os-token=ADMIN --os-url= --os-auth-type token_endpoint user list -f value -c Name00:35
IgorYozhikovthanx, will examine log files tomorrow - today :)00:36
*** bigdogstl has quit IRC00:37
IgorYozhikovthis is fresh packaged in deb pip freeze | grep openstackclient00:37
IgorYozhikovI packaged it locally00:37
dtroyer_zzwe just did 3.0.1 a few hours ago00:37
dtroyer_zzalso, os-client-config 1.20.1 is coming, if it isn't already, but (so far) isn't required to make things work00:38
*** bigdogstl has joined #openstack-keystone00:38
dtroyer_zzah, o-c-c 1.20.1 should be available now00:39
IgorYozhikovdtroyer_zz, updated via pip to 3.0.1 - works with --os-auth-type token_endpoint00:39
dtroyer_zzok, good to know, you have a work-around and we have another bug00:39
dtroyer_zzwould you mind making a new bug for this?  it's a completely different code base from the one you found from a while back.  tomorrow is fine if this is the end of your day00:40
IgorYozhikovit is 3.40am here O_o00:40
IgorYozhikovSo I need to create a new one instead of https://bugs.launchpad.net/python-openstackclient/+bug/161511000:41
openstackLaunchpad bug 1615110 in python-openstackclient "Incorrect usage message" [Undecided,New]00:41
*** bigdogstl has quit IRC00:41
dtroyer_zzyes please.  it's due to a different issue00:42
* dtroyer_zz is on a plane and didn't actually looka t that one, but I'm pretty sure this is new00:42
*** bigdogstl has joined #openstack-keystone00:43
IgorYozhikovdtroyer_zz, ok, will do after sleep :)00:43
*** bigdogst_ has joined #openstack-keystone00:47
*** bigdogstl has quit IRC00:48
*** bigdogst_ has quit IRC00:53
*** tqtran has quit IRC00:56
*** code-R has joined #openstack-keystone00:57
*** bigdogstl has joined #openstack-keystone01:01
*** bigdogstl has quit IRC01:06
*** sdake_ has joined #openstack-keystone01:06
*** gyee has quit IRC01:07
*** sdake has quit IRC01:10
*** jamielennox is now known as jamielennox|away01:11
*** jamielennox|away is now known as jamielennox01:16
*** sdake_ has quit IRC01:19
*** sdake has joined #openstack-keystone01:22
*** su_zhang has quit IRC01:26
jamielennoxanyone seen  py.warnings [req-ea36bb61-38ad-4a12-bea2-2a0db5f50fe7 - - - - -] /usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py:434:      SAWarning: Session's state has been changed on a non-active transaction - this state will be discarded.01:27
jamielennox  36   "Session's state has been changed on "01:27
jamielennox before?01:27
jamielennoxseeing it on my own devstack01:27
jamielennoxoh, nvm - restarting apache didn't help but restarting mysql did01:29
*** wangqun has joined #openstack-keystone01:31
*** davechen has joined #openstack-keystone01:34
*** hockeynut has quit IRC01:36
*** markvoelker has joined #openstack-keystone01:44
*** markvoelker_ has joined #openstack-keystone01:46
*** markvoelker has quit IRC01:50
*** EinstCra_ has joined #openstack-keystone02:01
*** EinstCra_ has quit IRC02:17
*** EinstCra_ has joined #openstack-keystone02:22
*** code-R_ has joined #openstack-keystone02:28
*** code-R has quit IRC02:31
*** EinstCra_ has quit IRC02:39
*** roxanagh_ has joined #openstack-keystone02:47
*** jamielennox is now known as jamielennox|away02:49
*** roxanagh_ has quit IRC02:52
*** tqtran has joined #openstack-keystone02:54
*** tqtran has quit IRC03:00
*** jamielennox|away is now known as jamielennox03:06
*** ayoung has quit IRC03:14
*** EinstCrazy has joined #openstack-keystone03:24
*** bigdogstl has joined #openstack-keystone03:31
openstackgerritSteve Martinelli proposed openstack/keystoneauth: Disables TCP_KEEPCNT when using Windows Subsystem for Linux  https://review.openstack.org/35745203:36
*** EinstCrazy has quit IRC03:38
*** bigdogstl has quit IRC03:41
*** bigdogstl has joined #openstack-keystone03:44
*** dikonoor has joined #openstack-keystone03:45
*** chrisshattuck has joined #openstack-keystone03:47
*** roxanagh_ has joined #openstack-keystone03:48
*** chrisshattuck has quit IRC03:50
*** chrisshattuck has joined #openstack-keystone03:51
*** roxanagh_ has quit IRC03:52
*** gagehugo has quit IRC03:53
*** chrisshattuck has quit IRC04:03
*** chrisshattuck has joined #openstack-keystone04:04
*** Jaison has joined #openstack-keystone04:15
*** markvoelker has joined #openstack-keystone04:21
*** markvoelker_ has quit IRC04:22
*** chrisshattuck has quit IRC04:24
*** chrisshattuck has joined #openstack-keystone04:24
*** ravelar has joined #openstack-keystone04:24
*** Jaison has quit IRC04:24
*** jraju has joined #openstack-keystone04:26
*** markvoelker has quit IRC04:28
*** EinstCrazy has joined #openstack-keystone04:30
*** su_zhang has joined #openstack-keystone04:39
*** EinstCrazy has quit IRC04:47
*** jamielennox is now known as jamielennox|away04:49
*** edtubill has quit IRC04:50
*** bigdogstl has quit IRC04:53
*** bigdogstl has joined #openstack-keystone04:53
*** bigdogstl has quit IRC04:56
*** bigdogstl has joined #openstack-keystone04:56
*** tqtran has joined #openstack-keystone04:57
*** bigdogstl has quit IRC05:01
*** tqtran has quit IRC05:01
*** roxanagh_ has joined #openstack-keystone05:02
*** roxanagh_ has quit IRC05:03
*** chrisshattuck has quit IRC05:05
*** chrisshattuck has joined #openstack-keystone05:06
*** jaosorior has joined #openstack-keystone05:10
*** bigdogstl has joined #openstack-keystone05:13
*** bigdogstl has quit IRC05:17
*** sdake_ has joined #openstack-keystone05:21
*** jamielennox|away is now known as jamielennox05:23
*** sdake has quit IRC05:24
*** su_zhang has quit IRC05:26
*** su_zhang has joined #openstack-keystone05:26
*** markvoelker has joined #openstack-keystone05:29
*** markvoelker has quit IRC05:39
*** richm has quit IRC05:40
*** sdake_ has quit IRC05:41
*** roxanaghe has quit IRC05:55
*** roxanaghe has joined #openstack-keystone05:55
*** roxanagh_ has joined #openstack-keystone06:04
*** gagehugo has joined #openstack-keystone06:05
*** roxanagh_ has quit IRC06:08
*** jamielennox is now known as jamielennox|away06:11
*** pcaruana has joined #openstack-keystone06:14
*** woodster_ has quit IRC06:19
*** chrisshattuck has quit IRC06:29
*** EinstCrazy has joined #openstack-keystone06:32
*** markvoelker has joined #openstack-keystone06:36
*** jaugustine has quit IRC06:38
*** EinstCrazy has quit IRC06:38
*** markvoelker has quit IRC06:42
*** EinstCra_ has joined #openstack-keystone06:43
*** GB21 has joined #openstack-keystone06:47
*** su_zhang has quit IRC06:48
*** su_zhang has joined #openstack-keystone06:48
adriantthis is probably a silly question, but where in keystone client is this api call: http://developer.openstack.org/api-ref/identity/v3/?expanded=#list-projects-for-user06:50
*** su_zhang has quit IRC06:52
openstackgerritDave Chen proposed openstack/keystone: Fix credential update to ec2 type  https://review.openstack.org/35795006:58
*** tqtran has joined #openstack-keystone06:59
*** EinstCra_ has quit IRC07:01
*** tqtran has quit IRC07:03
*** mdurrant_ has quit IRC07:04
*** mdurrant_ has joined #openstack-keystone07:04
*** EinstCrazy has joined #openstack-keystone07:05
*** tesseract- has joined #openstack-keystone07:18
*** EinstCrazy has quit IRC07:24
*** EinstCrazy has joined #openstack-keystone07:25
henrynashadriant: do you mean the keystoneclient library or the keystoneclient cli?07:29
*** asettle has joined #openstack-keystone07:33
*** dikonoor has quit IRC07:35
*** dikonoor has joined #openstack-keystone07:36
*** EinstCra_ has joined #openstack-keystone07:36
adrianthenrynash: client library, but mainly curious how exactly openstackclient calls it07:38
adriantas it seems to just use list directly, but according to the policy file that shouldn't work unless you're admin07:38
*** markvoelker has joined #openstack-keystone07:38
*** EinstCrazy has quit IRC07:39
*** markvoelker has quit IRC07:43
adriantplus... openstackclient throws a fit when you pass a username rather than an id as it needs admin to parse that username to an id.07:43
adriantSo to get user project list using the openstackclient you need to get your id by token issue and then use that.07:44
*** GB21 has quit IRC07:45
*** roxanagh_ has joined #openstack-keystone07:52
*** adriant_ has joined #openstack-keystone07:54
*** rcernin has quit IRC07:56
*** roxanagh_ has quit IRC07:56
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-keystone08:01
*** openstackgerrit has quit IRC08:03
*** openstackgerrit has joined #openstack-keystone08:04
*** adriant_ has quit IRC08:07
*** adriant__ has joined #openstack-keystone08:11
*** markvoelker has joined #openstack-keystone08:39
*** markvoelker has quit IRC08:44
*** ravelar has quit IRC08:50
*** adriant__ has quit IRC08:53
IgorYozhikovdtroyer_zz, as promised - https://bugs.launchpad.net/python-openstackclient/+bug/161598808:55
openstackLaunchpad bug 1615988 in python-openstackclient "[osclient v3.x.x]Default auth type is not detected in case when os-token and os-url is used " [Undecided,New]08:55
*** code-R_ has quit IRC08:56
henrynashadriant: sorry, was afk for a while....so is the need to be able to list your own projects (i.e. the username is you), or to list the projects for another user?09:06
*** code-R has joined #openstack-keystone09:14
*** d0ugal has quit IRC09:15
*** d0ugal has joined #openstack-keystone09:16
openstackgerritzhufl proposed openstack/keystone: Remove unnecessary __init__  https://review.openstack.org/35906309:17
*** willise has joined #openstack-keystone09:31
*** willise has left #openstack-keystone09:32
*** EinstCra_ is now known as EinstCrazy09:36
*** roxanagh_ has joined #openstack-keystone09:40
*** markvoelker has joined #openstack-keystone09:40
*** markvoelker has quit IRC09:44
*** roxanagh_ has quit IRC09:45
*** dikonoor has quit IRC09:51
*** dikonoor has joined #openstack-keystone09:52
*** EinstCrazy has quit IRC09:57
*** EinstCrazy has joined #openstack-keystone09:57
*** EinstCra_ has joined #openstack-keystone09:58
*** EinstCra_ is now known as EinstCrazy_09:59
*** tqtran has joined #openstack-keystone10:01
*** EinstCrazy has quit IRC10:02
*** wangqun has quit IRC10:03
*** code-R_ has joined #openstack-keystone10:03
samueldmqmorning keystone10:04
bretonsamueldmq: o/10:04
samueldmqbreton: hi10:05
*** tqtran has quit IRC10:05
*** code-R has quit IRC10:06
*** richm has joined #openstack-keystone10:07
*** EinstCrazy_ has quit IRC10:28
*** ntpttr has quit IRC10:30
*** ntpttr has joined #openstack-keystone10:35
samueldmqstevemar: keystonemiddleware release is proposed in patch 35910610:43
samueldmqstevemar: I am waiting on 357633 and 357452 to propose keystoneauth10:43
samueldmqpatch 35763310:43
samueldmqchange 35763310:44
samueldmqbot's broken10:44
*** mnikolaenko_ has quit IRC10:47
*** davechen has left #openstack-keystone10:51
*** rodrigods has quit IRC10:59
*** rodrigods has joined #openstack-keystone10:59
*** sheel has joined #openstack-keystone11:02
*** jpena is now known as jpena|lunch11:06
*** markvoelker has joined #openstack-keystone11:09
*** ravelar has joined #openstack-keystone11:18
*** markvoelker has quit IRC11:20
openstackgerritRodrigo Duarte proposed openstack/python-keystoneclient: Support domain-specific configuration management  https://review.openstack.org/35877011:22
*** ravelar has quit IRC11:23
*** thiagolib has quit IRC11:23
*** thiagolib has joined #openstack-keystone11:24
*** roxanagh_ has joined #openstack-keystone11:28
*** roxanagh_ has quit IRC11:32
*** amakarov_away is now known as amakarov11:36
amakarovdstanek, I'm online11:37
*** rcernin has joined #openstack-keystone11:43
*** jaosorior has quit IRC11:50
*** asettle has quit IRC11:51
*** jaosorior has joined #openstack-keystone11:51
*** wangqun has joined #openstack-keystone11:51
*** lamt has joined #openstack-keystone12:02
*** mordred has quit IRC12:10
*** Gorian has quit IRC12:12
*** Gorian has joined #openstack-keystone12:12
*** mordred has joined #openstack-keystone12:15
*** pauloewerton has joined #openstack-keystone12:18
*** david_cu has joined #openstack-keystone12:18
*** rcernin has quit IRC12:23
*** rcernin has joined #openstack-keystone12:28
*** jed56 has joined #openstack-keystone12:31
*** nkinder has joined #openstack-keystone12:31
*** edmondsw has joined #openstack-keystone12:40
*** jpena|lunch is now known as jpena12:40
*** asettle has joined #openstack-keystone12:40
dstanekamakarov: heya12:46
amakarovdstanek, o/12:55
dstanekamakarov: i'm curious about your caching approach and how it compares to just hard dropping the keys like i am doing12:56
*** chlong has quit IRC12:56
*** woodster_ has joined #openstack-keystone12:56
amakarovdstanek, your case is faster: it doesn't care about timestamp checks12:57
dstanekamakarov: right, the two things that bother me about the timestamp approach is that it's possible for clock skew to be a problem and to drop invalidations12:58
dstanekunlikely that they would drop, but possible12:58
amakarovdstanek, but one thing still looks weird: dogpile.cache monkeypatching12:58
*** guoshan has joined #openstack-keystone12:59
dstanekamakarov: i have three different implementations based off of your and zzzeek's suggestions12:59
dstaneki'd love to get support in dogpile for some of this stuff because i can't see how anyone could use dogpile without it13:00
dstanekamakarov: https://review.openstack.org/#/c/358826/1/keystone/common/cache/core.py13:01
dstaneki'm going to clean that one up since i think it's probably the best13:01
dstanekfix the pep8 and duplication issues, etc.13:01
amakarovdstanek, the last one?13:02
*** tqtran has joined #openstack-keystone13:03
dstanekamakarov: yeah, uses a key mangler as zzzeek suggested and an invalidation strategy as you suggested13:03
*** rcernin has quit IRC13:03
amakarovdstanek, I like this one13:04
amakarovdstanek, why do you add "this should last forever" everywhere? :)13:05
dstanekamakarov: those are reminders for me to make sure the cache expiration for the region won't apply to those keys13:06
*** tqtran has quit IRC13:07
amakarovdstanek, I'm curios: why use uuid4 everytime and not just random? Lower chance of collision?13:11
*** chlong has joined #openstack-keystone13:12
*** wangqun has quit IRC13:14
dstanekamakarov: no particular reason. doing a 'random.randint(0, sys.maxint)' would probably be faster, but i was using uuid in the tests already13:14
dstanekthe chance of collision is so tiny because it's only comparing a few random bits of data, unlike our database that has 10s of 1000s of rows13:16
*** roxanagh_ has joined #openstack-keystone13:16
dstanekthe biggest chance of collision is if the same random number comes up as a past one where there is still some old data cached13:17
amakarovdstanek, I'll be happy to support this patch once you deal with Jenkins :) I like it.13:19
*** roxanagh_ has quit IRC13:21
*** julim has joined #openstack-keystone13:21
dstanekamakarov: i actually think the monkey patching wasn't useful anyway. it was a holdover from before i created a keystone specific create_region()13:22
amakarovdstanek, ++13:23
*** bigdogstl has joined #openstack-keystone13:24
*** david-lyle has joined #openstack-keystone13:24
*** tonytan4ever has joined #openstack-keystone13:24
*** gagehugo_ has joined #openstack-keystone13:32
*** gagehugo has quit IRC13:32
*** gagehugo_ has quit IRC13:32
*** gagehugo has joined #openstack-keystone13:33
*** sdake has joined #openstack-keystone13:36
*** sheel has quit IRC13:36
*** sdake_ has joined #openstack-keystone13:37
*** jraju has quit IRC13:40
*** sdake has quit IRC13:41
*** ayoung has joined #openstack-keystone13:42
*** ChanServ sets mode: +v ayoung13:42
*** bigdogstl has quit IRC13:43
*** ravelar has joined #openstack-keystone13:46
*** eandersson_ has quit IRC13:47
*** raildo has joined #openstack-keystone13:51
*** tonytan4ever has quit IRC13:56
*** eandersson has joined #openstack-keystone13:56
*** guoshan has quit IRC13:57
*** bigdogstl has joined #openstack-keystone13:57
*** tqtran has joined #openstack-keystone14:02
*** bigdogstl has quit IRC14:02
*** pcaruana has quit IRC14:02
*** david-lyle has quit IRC14:04
*** tonytan4ever has joined #openstack-keystone14:08
*** ddieterly has joined #openstack-keystone14:12
*** code-R has joined #openstack-keystone14:14
*** code-R_ has quit IRC14:14
*** pcaruana has joined #openstack-keystone14:17
*** edtubill has joined #openstack-keystone14:17
openstackgerritRon De Rose proposed openstack/keystone: Relax the requirement for mappings to result in group memberships  https://review.openstack.org/35811114:20
openstackgerritMerged openstack/keystoneauth: Disables TCP_KEEPCNT when using Windows Subsystem for Linux  https://review.openstack.org/35745214:22
*** ravelar has quit IRC14:28
*** ravelar has joined #openstack-keystone14:28
*** david-lyle has joined #openstack-keystone14:28
*** adam_g has quit IRC14:29
*** spedione|AWAY is now known as spedione14:30
*** jaosorior is now known as jaosorior_away14:32
dstanekwell me too! o/14:33
*** adam_g has joined #openstack-keystone14:34
*** adam_g has quit IRC14:34
*** adam_g has joined #openstack-keystone14:34
*** sdake_ has quit IRC14:35
stevemardstanek: :O14:36
*** vern has quit IRC14:36
stevemarsamueldmq: ^ you're all clear to release keystoneauth14:36
*** ezpz has joined #openstack-keystone14:39
*** jaugustine has joined #openstack-keystone14:39
samueldmqstevemar: hi14:39
samueldmqstevemar: don't we want https://review.openstack.org/#/c/357633/ in too ?14:39
samueldmqstevemar: crinkle left a question there I don't have an answer to, I think it is a standard but  I am not 100% sure14:40
*** sdake has joined #openstack-keystone14:41
stevemarsamueldmq: ideally we want that in, but it's tuesday; jamie has to answer that and won't come online until late afternoon, and that is too late to release in the day; and i'd rather not until wednesday14:42
stevemarsamueldmq: no reason to rush and merge it, we've lived without it, and will continue to live without it :)14:42
samueldmqstevemar: yes, it should be fine, that's not critical14:42
samueldmqstevemar: ++14:42
amakarovstevemar, greetings! I've fixed tests and answered questions here: https://review.openstack.org/#/c/309146/ Would you please take a look again?14:47
stevemaramakarov: will do!14:48
*** tqtran has quit IRC14:49
*** bigdogstl has joined #openstack-keystone14:52
openstackgerritMerged openstack/keystone: Shadowing a nonlocal_user incorrectly creates a local_user  https://review.openstack.org/35797914:54
*** slberger has joined #openstack-keystone14:54
*** david-lyle has quit IRC14:54
openstackgerritMikhail Nikolaenko proposed openstack/keystone: [WIP] Move fernet utils to backend  https://review.openstack.org/35649914:56
*** bigdogstl has quit IRC14:57
*** hockeynut has joined #openstack-keystone14:58
*** code-R_ has joined #openstack-keystone14:58
crinklesamueldmq: stevemar I didn't mean to hold up a release, I was just curious about it15:00
*** code-R has quit IRC15:01
*** vern has joined #openstack-keystone15:03
samueldmqcrinkle: that's a good question you left in the review15:04
samueldmqcrinkle: there is no rush to have that in this release, as stevemar said15:04
samueldmqrelease is proposed already https://review.openstack.org/#/c/35926115:04
*** edtubill has quit IRC15:07
AlexOughtonHey stevemar samueldmq and anyone else involved: Thanks for helping clean up my submission for the TCP_KEEPCNT thing and getting that pushed through! :-)15:15
openstackgerritDolph Mathews proposed openstack/keystone: Doc fix: "keystone-manage upgrade" is not a thing  https://review.openstack.org/35928115:18
dolphmquick doc fix for rolling upgrades! ^15:19
SamYaplewell let this one slide dolphm15:22
SamYaplejust this once15:22
openstackgerritDolph Mathews proposed openstack/keystone: Doc fix: license rendered in published doc  https://review.openstack.org/35928415:22
*** david-lyle has joined #openstack-keystone15:23
openstackgerritTobias Diaz proposed openstack/python-keystoneclient: Catch MemoryError when logging huge responses Closes-bug: 1616105  https://review.openstack.org/35929215:28
openstackbug 1616105 in python-keystoneclient "Request of large files raises a MemoryError due to logging" [Undecided,In progress] https://launchpad.net/bugs/1616105 - Assigned to Tobias Diaz (int-0)15:28
samueldmqAlexOughton: hey, you're welcome15:31
*** david-lyle has quit IRC15:33
openstackgerritDolph Mathews proposed openstack/keystone: Doc fix: license rendered in published doc  https://review.openstack.org/35928415:35
EmilienMstevemar: hey !15:36
EmilienMwhat is supposed to do https://review.openstack.org/358969 ?15:36
stevemarEmilienM: hey15:36
stevemarEmilienM: just making sure that you guys aren't gonna break if we change the upper-constraints to 3.0.115:36
EmilienMour CI relies on packaging (RDO or UCA), so depends-on won't work between puppet-* and python projects, except Tempest.15:36
EmilienMit is not something we can test15:36
stevemarEmilienM: :(15:36
EmilienMbut, you can ask on #rdo, we have a mechanism to test it super easily15:37
EmilienMin RDO Ci15:37
stevemardolphm: but "upgrade" is better than "db_sync" :P15:44
stevemarwe clearly need an alias15:44
*** code-R_ has quit IRC15:45
*** code-R has joined #openstack-keystone15:46
*** tesseract- has quit IRC15:46
*** bigdogstl has joined #openstack-keystone15:46
*** dikonoor has quit IRC15:49
*** ddieterly is now known as ddieterly[away]15:50
*** bigdogstl has quit IRC15:53
*** ddieterly[away] is now known as ddieterly15:54
*** pcaruana has quit IRC15:56
*** nishaYadav has joined #openstack-keystone15:58
*** jaosorior_away is now known as jaosorior16:00
*** david-lyle has joined #openstack-keystone16:01
*** Ephur has joined #openstack-keystone16:02
*** gyee has joined #openstack-keystone16:03
stevemaredmondsw: is https://bugs.launchpad.net/python-openstackclient/+bug/1616104 related to the 3.0.1 release, or just in general16:03
openstackLaunchpad bug 1616104 in python-openstackclient "openstack user list --project <p> not showing effective assignments" [Undecided,New]16:04
edmondswstevemar found in 2.6.0, haven't tried 3.0.0 yet16:04
edmondswstevemar hadn't even realized yet that there was a 3.0.1... wasn't 3.0.0 just yesterday? what caused the respin so fast?16:05
stevemaredmondsw: 3.0.0 didn't work lol16:05
stevemaredmondsw: we were caught between os-client-config releases16:05
stevemarits all good now :P16:05
stevemaredmondsw: but we shuffled things around for listing role assignments16:06
edmondswstevemar, ok, so I'll have to try again on 3.0.1 once I can get that setup... probably not today16:07
*** michauds has joined #openstack-keystone16:08
*** pcaruana has joined #openstack-keystone16:09
*** nkinder has quit IRC16:11
stevemaranyone want to look at https://review.openstack.org/#/c/343028/ ? i've reviewed it a few times now, it's one of our last blueprints16:14
stevemarcrinkle dolphm samueldmq edmondsw ^16:14
stevemarrderose: ^16:15
edmondswstevemar I'll try to look this afternoon16:17
*** code-R has quit IRC16:18
*** jaosorior has quit IRC16:18
* crinkle looks16:19
*** bigdogstl has joined #openstack-keystone16:23
rderosestevemar: looking...16:25
*** bigdogstl has quit IRC16:27
*** chrisshattuck has joined #openstack-keystone16:29
openstackgerritLance Bragstad proposed openstack/keystone: Implement encryption of credentials at rest  https://review.openstack.org/35561816:29
dolphmstevemar: +2/+A16:30
lbragstaddolphm did the mysql trigger - curious for feedback on it16:30
samueldmqstevemar: will do16:30
lbragstadstill looking into the equivalent trigger for sqlite and postgresql16:30
dolphmlbragstad: link?16:30
samueldmqstevemar: too late, approved :-)16:30
lbragstaddolphm https://review.openstack.org/#/c/355618/16:30
dstanekamakarov: i got rid of the comment you mentioned earlier by creating a region just for invalidation keys16:31
lbragstaddolphm I responded to all comments on patchset 11 and fixed most of them in the latest patch16:31
dolphmlbragstad: it's the UPDATE case that needs to be conditional - not BEFORE INSERT (which the old triggers were good for)16:31
dolphmlbragstad: but the conditional looks correct for the BEFORE UPDATE case16:32
lbragstaddolphm cool - updating16:32
dolphmlbragstad: so, you'll need 6 triggers total16:33
dolphmlbragstad: and don't forget to tear down the second trigger in contract16:33
*** nishaYadav has quit IRC16:33
dolphmlbragstad: ... and in a BEFORE UPDATE, you'll also have both a NEW. and an OLD. to deal with16:33
dolphmlbragstad: BEFORE UPDATE only has a NEW row16:33
edmondswstevemar dolphm I think the release notes still need work there16:34
lbragstaddolphm we only care if NEW.key_hash and NEW.encrypted_blob are populated though, right?16:34
edmondswI mean https://review.openstack.org/#/c/34302816:35
*** woodburn has quit IRC16:36
edmondswbreton ^16:37
dolphmlbragstad: before the migration, only blob will be populated, right?16:37
*** BjoernT has joined #openstack-keystone16:38
dolphmlbragstad: and the old release will only try to populate blob16:38
dolphmlbragstad: the new release will only try to populate encrypted_blob16:38
dolphmlbragstad: (and key hash)16:38
dolphmlbragstad: both of which are operations you need to detect and block16:38
*** jraju has joined #openstack-keystone16:38
dolphmlbragstad: but you need to allow the --migrate process to run, which will be setting encrypted_blob and key_hash on rows where the blob already exists16:38
dolphmlbragstad: using UPDATE16:38
dolphmlbragstad: whereas nothing will be INSERTing, so you can block all of those16:39
dolphmlbragstad: so, the complex trigger (BEFORE UPDATE) will basically be checking to see if this is --migrate running, and if so, allow it, else signal16:39
*** jraju has quit IRC16:39
*** ravelar has quit IRC16:40
dolphmstevemar: still going to be AFK for the meeting?16:40
*** ravelar has joined #openstack-keystone16:40
lbragstaddolphm the migrate process will always populate key_hash and encrypted_blob16:40
lbragstadif key_hash or encrypted_blob are null then signal16:41
bretonedmondsw: dolphm: thanks, i've wf-1 the change16:41
dolphmlbragstad: then you'd be allowing the legacy app to keep writing :-/16:41
* breton spoiled wf+116:41
edmondswbreton sorry... and thanks16:42
dolphmbreton: edmondsw: oops, sorry, i only reviewed the diff between 12 and 1316:42
lbragstaddolphm the legacy app defaults key_hash to null if it's not populated16:42
lbragstadah - nevermind... i see16:43
dolphmlbragstad: oh, sorry, then you'll be allowing newton app to write16:43
dolphmlbragstad: and mitaka won't be able to read it16:43
dolphmlbragstad: this is totally not confusing at all.16:43
lbragstadwhat happens if encrypted_blob isn't supplied to a schema that doesn't allow it to be nullable?16:43
dolphmlbragstad: that line doesn't matter because you're actually defining it to be nullable here https://review.openstack.org/#/c/355618/12/keystone/common/sql/expand_repo/versions/002_add_key_hash_and_encrypted_blob_to_credential.py16:44
dolphmlbragstad: so, your model does not match the actual schema16:45
dolphmlbragstad: unless you do an alter column in the contract -- which we could allow, i think16:45
dolphmlbragstad: but only in mysql and postgres16:45
*** asettle has quit IRC16:48
*** asettle has joined #openstack-keystone16:49
openstackgerritBoris Bobrov proposed openstack/keystone: Add mapping_populate command  https://review.openstack.org/34302816:49
*** asettle has joined #openstack-keystone16:49
*** awayne has quit IRC16:49
*** asettle has quit IRC16:50
*** amakarov is now known as amakarov_away16:50
*** ctracey has quit IRC16:53
*** ctracey has joined #openstack-keystone16:54
openstackgerritLance Bragstad proposed openstack/keystone: Implement encryption of credentials at rest  https://review.openstack.org/35561816:54
*** woodburn has joined #openstack-keystone16:55
*** jed56 has quit IRC16:55
*** nishaYadav has joined #openstack-keystone16:56
lbragstadgoing to grab lunch quick16:57
*** chrisshattuck has quit IRC16:58
*** nkinder has joined #openstack-keystone17:05
*** Guest25180 is now known as med_17:06
*** med_ has joined #openstack-keystone17:06
*** med_ is now known as medberry17:06
*** Gorian|work has joined #openstack-keystone17:06
*** medberry is now known as med_17:06
*** ddieterly is now known as ddieterly[away]17:10
*** bigdogstl has joined #openstack-keystone17:15
openstackgerritMorgan Fainberg proposed openstack/keystone: Filter data when deserializing RevokeEvents  https://review.openstack.org/35887217:17
*** jrist has quit IRC17:18
*** bigdogstl has quit IRC17:20
*** jaugustine_ has joined #openstack-keystone17:26
*** jaugustine has quit IRC17:27
*** jaugustine_ is now known as jaugustine17:27
*** gagehugo_ has joined #openstack-keystone17:27
*** su_zhang has joined #openstack-keystone17:29
*** hockeynut has quit IRC17:33
*** tqtran has joined #openstack-keystone17:35
openstackgerritayoung proposed openstack/keystone-specs: Flag to bypass expiry and revocation check on token validation  https://review.openstack.org/35813117:36
*** tqtran has quit IRC17:39
*** code-R has joined #openstack-keystone17:42
*** code-R_ has joined #openstack-keystone17:44
*** Ephur has quit IRC17:45
openstackgerritDavid Stanek proposed openstack/keystone: Distributed cache namespace to invalidate regions  https://review.openstack.org/34970417:46
*** jrist has joined #openstack-keystone17:46
*** code-R has quit IRC17:47
*** nishaYadav has quit IRC17:48
*** nisha_ has joined #openstack-keystone17:48
*** su_zhang has quit IRC17:51
*** bigdogstl has joined #openstack-keystone17:51
*** su_zhang has joined #openstack-keystone17:51
*** zhugaoxiao has joined #openstack-keystone17:53
*** Administrator__ has quit IRC17:55
*** bigdogstl has quit IRC17:58
dolphmbknudson: bandit should be downvoting lbragstad's patch for potential SQL injection vulnerability around L90 https://review.openstack.org/#/c/355618/14/keystone/common/sql/expand_repo/versions/002_add_key_hash_and_encrypted_blob_to_credential.py,unified17:59
dolphmbknudson: something we're going to have to be super careful about now that we're writing more manual SQL18:00
henrynashdolphm, bknudson: I did write some SQL injection detection code a while back...18:00
*** su_zhang has quit IRC18:00
*** su_zhang has joined #openstack-keystone18:01
*** jamielennox|away is now known as jamielennox18:01
*** su_zhang has quit IRC18:01
*** su_zhang has joined #openstack-keystone18:02
*** su_zhang has quit IRC18:02
*** su_zhang has joined #openstack-keystone18:02
*** nisha_ has quit IRC18:03
*** shaleh has joined #openstack-keystone18:06
*** nishaYadav has joined #openstack-keystone18:06
*** nishaYadav has quit IRC18:10
openstackgerritRichard Avelar proposed openstack/keystone: POC sql query revoked tokens  https://review.openstack.org/35937118:12
*** hockeynut has joined #openstack-keystone18:13
*** nishaYadav has joined #openstack-keystone18:13
*** Ephur has joined #openstack-keystone18:17
lbragstadhenrynash ping to see if you have time to talk about the rolling upgrade testing stuff after the meeting?18:21
openstackgerritayoung proposed openstack/keystone: Modify sql banned operations for each of the new repos  https://review.openstack.org/35872318:21
*** su_zhang has quit IRC18:41
*** su_zhang has joined #openstack-keystone18:41
*** code-R_ has quit IRC18:41
*** code-R has joined #openstack-keystone18:42
*** ddieterly[away] is now known as ddieterly18:44
jamielennoxso why are we doing DB triggers now?18:51
henrynashlbragstad: you shouldn't do self.upgrade(self.max_version) in your test, but you should do self.upgrade(2)18:51
*** amakarov has joined #openstack-keystone18:51
jamielennoxi thought DB triggers were outside our ability with cross sql requirements18:51
lbragstadhenrynash got it - the interesting part it that it doesn't fail my test18:51
lbragstadhenrynash it fails here - https://github.com/openstack/keystone/blob/0cd732b2b0d3e18cbdbceecf66a83cd378c27717/keystone/tests/unit/test_sql_banned_operations.py#L21118:52
*** su_zhang has quit IRC18:52
henrynashlbragstad: well, since you are 2 then max_version is 2, so you are OK...but as soon as we add a 3rd, your test might be wroung18:52
ayoungdolphm, so, I'll add the grace period as part of that review.18:52
ayoungthe spec review, that is18:52
henrynashlbragstad: but that's not why your is failing18:52
* lbragstad henrynash this is the specific failure - http://cdn.pasteraw.com/6zaqyxiupmfikpeqbzn86wwicgqcni318:53
lbragstadwhich looks like it fails because the credential table isn't created18:53
henrynashlbragstad: ah!!18:56
henrynashlbragstad: this is failing in sql_banned_operations...not sql_upgrade....18:56
lbragstadhenrynash yep!18:56
henrynashlbragstad: that ISN'T fixed in master yet....18:56
lbragstadhenrynash is that fixed somewhere in https://review.openstack.org/#/c/358723/1/keystone/tests/unit/test_sql_banned_operations.py ?18:57
henrynashlbragstad: rebase on https://review.openstack.org/#/c/358723/118:57
henrynashlbragstad: yep, that ensures the early repos are migrated up first18:57
henrynashlbragstad: sorry, hadn't twiged that it was sql_banned_operations that was failing18:58
samueldmqah, just as a note18:58
jamielennoxshaleh: i jumped ahead and did the user-agent change: https://review.openstack.org/#/c/357633/18:58
*** markvoelker has joined #openstack-keystone18:58
samueldmqkeystonemiddleware and keystoneauth have been released today18:58
samueldmqdolphm: ^18:58
lbragstadhenrynash ok - that makes sense18:58
jamielennoxoh, damn - i was hoping to get that usre_agent patch in a next release18:59
samueldmqand 1 minutes left18:59
jamielennoxsamueldmq: we're not in a meeting :p18:59
* samueldmq facepalm18:59
* samueldmq goes afk18:59
dolphmsamueldmq: you're excused19:01
*** markvoelker has quit IRC19:01
shalehjamielennox: thanks. blah. Work and stuff.19:02
shalehjamielennox: I had made it to about half of the unit testing and got pulled away19:03
*** tqtran has joined #openstack-keystone19:04
*** ddieterly is now known as ddieterly[away]19:04
jamielennoxshaleh: no worries, turns out i wanted something like that for another reason and so didn't think you'd mind19:04
jamielennoxi was hoping to get it in that last ksa release though to start pushing people towards it19:04
openstackgerrithenry-nash proposed openstack/keystone: Update developer docs for new rolling upgrade repos  https://review.openstack.org/35938319:05
*** code-R has quit IRC19:07
openstackgerrithenry-nash proposed openstack/keystone: Update developer docs for new rolling upgrade repos  https://review.openstack.org/35938319:07
jamielennoxwhy are we using database triggers now?19:08
jamielennoxwhy isn't https://review.openstack.org/#/c/357789/3/keystone/common/sql/expand_repo/versions/002_mysql_upgrade.sql a code fix, a migration and a default19:09
rodrigodsjamielennox, had the same question when i saw it19:09
jamielennoxwe've alwas considered triggers outside the cross-sql platform we support19:10
lbragstadhenrynash sweet - in order to cleanly rebase on your patch I had to rebase it19:11
henrynashlbragstad: ok, thx19:11
lbragstadhenrynash mind if I push a new patch set?19:11
henrynashlbragstad: go for it19:11
lbragstadhenrynash for https://review.openstack.org/#/c/358723/219:11
openstackgerritLance Bragstad proposed openstack/keystone: Modify sql banned operations for each of the new repos  https://review.openstack.org/35872319:11
openstackgerritLance Bragstad proposed openstack/keystone: Implement encryption of credentials at rest  https://review.openstack.org/35561819:11
*** tonytan_brb has joined #openstack-keystone19:11
henrynashjamielennox, rodrigods: so on https://review.openstack.org/#/c/357789/3 I did originally write it without triggers, but dolphm, redrose were keen that we keep to strict use of the three new phases19:12
openstackgerrithenry-nash proposed openstack/keystone: Fix issue of password created_at being left as nullable  https://review.openstack.org/35778919:13
*** spzala has joined #openstack-keystone19:13
shalehjamielennox: looks a lot like my code. I saved the generated user_agent on self so it did not need to be examined on every request.19:13
jamielennoxhenrynash: to me this is where we have to do if db_version>X also set a timestamp on the query19:14
*** tonytan4ever has quit IRC19:14
jamielennoxroll code, migrate data, add default19:14
jamielennoxor just do it in migrate if you can do a default and NULL column, i don't think you an19:15
henrynashjamielennox: agreed....although th problem of the original patch was rderose couldn't find a way of having a common server default for the date that worked across all three DBs19:15
jamielennoxshaleh: i save the user_agent on self, just i don't generate it till first needed19:17
henrynashjamielennox: ...and (almost) every migration from now on will have triggers anyway...so we better get used to them19:18
*** nishaYadav has quit IRC19:18
jamielennoxwhy will every migration have triggers?19:18
*** nishaYadav has joined #openstack-keystone19:18
jamielennoxshouldn't sqlalchemy-migrate handle the common DEFAULT=NOW()19:19
jamielennoxand sqlalchemy handle parsing a datetime19:19
shalehtriggers? triggers do not play nice with multi-master Galera right?19:19
notmorganmordred: ^ cc sql trigger talking is scaring me.19:19
henrynashjamielennox: ..cause that's how rolling upgrades are implemented as per dolphm's simplificaiton19:19
jamielennoxto me triggers are that magic you invoke only when you have one running database you understand completely19:20
shalehjamielennox: in request() you set 'user_agent = headers.setdefault()' instead of 'self.user_agent = headers.setdefault()'19:20
rderosejamielennox: it should have, but doesn't work for alter table19:20
rderosejamielennox: works when create table19:20
*** pcaruana has quit IRC19:20
mordrednotmorgan: I didn't do it19:21
jamielennoxshaleh: oh, yea, i save the generated portion, but request() will be used by many different clients so i can't save the entire user_agent string because it will change19:21
* mordred scans backscroll19:21
*** nishaYadav has quit IRC19:21
notmorganmordred: no.. but you can say "sql + triggers" and appropriate response to said "thing"19:21
rderosejamielennox henrynash: triggers seem to be the most common way to deal with rolling upgrades19:21
mordrednotmorgan: yah. don't do triggers19:21
jamielennoxrderose: is that a sqlalchemy migrate bug? i think openstack owns that19:21
notmorgancause everytime someone brings up triggers... it scares me.19:21
shalehjamielennox: I am a little confused then. What is the difference between "app" and "client" here?19:21
notmorgan^ keystone folks, see what mordred said, trust him on that.19:21
jamielennoxshaleh: app=nova, client=glanceclient19:22
shalehrderose: we took over sqlalchemy-migrate19:22
notmorgando not do triggers19:22
mordrednotmorgan: what is the use case people are wanting to solve with them?19:22
notmorganmordred: does it really matter? it involves orm/migrations/zero downtime upgrades iirc.19:22
shalehmodred, notmorgan: rain the -2's19:22
notmorganmordred: but.. does it really matter with an ORM?19:22
jamielennoxshaleh: the client part of that particularly will change as nova talks to differnt services19:23
rderosejamielennox shaleh: cool19:23
notmorganmordred: my answer is the same: RUN AWAY, RUN AWAY19:23
mordrednotmorgan: yeah. not really - a sideband process is gonna work much better forhow all of our stuff is organized19:23
shalehjamielennox: ah. right. I remember us talking about that now.19:23
mordrednotmorgan: also, I want to say nova already has a pretty good story for this, so whatever they're doing is  likely what keystone should do19:24
*** ddieterly[away] is now known as ddieterly19:24
notmorganmordred: there are some wider concerns with keystone that we've run into (edge cases) but generally I agree.19:24
notmorganshaleh: sql triggers are going to def. be worthy of me dreding up a -2 :)19:25
dstanekmordred: notmorgan: the idea behind triggers was to give us a nice migration path that is near 100% uptime without having to drag along too much code19:25
notmorgangod... i can't type19:25
dstanekmordred: notmorgan: allows us to add transition logic to an N-1 release after the fact19:26
notmorgandstanek: there are so many issue with triggers. if you run a DB and tightly control all aspects it might work.19:26
notmorgandstanek: but we support an ORM. and many DB backends (many = more than MySQL), + galera in the mix. it is going to be rough going to make sure it all works19:26
jamielennoxalright, well i feel like i've kicked this hornets nest in the right direction but i want to catch some more sleep before the sun comes up19:27
dstaneki think the plan was to support just MySQL and Postgres at this point19:27
*** gyee has quit IRC19:27
jamielennoxbut i'm very skeptical that triggers are the right way to do this when we don't control the db19:27
notmorganright. that is going to make your life hard.19:27
henrynashnotmorgan, dstanek: we only install them temporarily between the expand and contract phases of an upgrade19:28
dstanekhenrynash: exactly19:28
dstanekit allows for incremental migration of data while everything is up and running19:28
lbragstaddstanek and sqlite until your patch goes through19:28
jamielennoxthat should be something we test for in our code19:28
notmorganhenrynash: you should talk more in depth with mordred about that - really. Triggers result in tears in my experience.19:28
jamielennoxwe are going to need that abstraction layer of if db_version X in our backends19:28
*** hockeynut has quit IRC19:28
dstaneklbragstad: i think we'll always need our unit tests to run in sqlite19:28
henrynashjamielennox: that is the alternative19:29
lbragstaddstanek so that makes sqlite, mysql, and postgres, right?19:29
notmorganthis is a case, that as much as I dislike it, I am going to advocate: Carry code that does the work with the schema intelligently19:29
bknudsonthere's a library for that abstraction layer - oslo.versionedobjects19:29
mordrednotmorgan: ++19:29
jamielennoxbknudson: there's not a lot of documentation for that last i checked19:30
mordredmanaging the trigger definitions and ddl in SQL for each backend is going to be clunky and different - whereas doing it closer to the  model/orm layer will keep it in the DB language already being used19:30
notmorgando not lean on SQL backends because it will end in tears because something will be subtly different between MySQL, Postgres, and Oh hi i used <other engine not supported bug it used to work> user19:30
notmorganmordred: ++19:30
dstanekbknudson: that's terrible from what i've seen19:30
bknudsonjamielennox: the code is the documentation!19:30
notmorganyou're going to end up writing DDL specific triggers... might as well drop the ORM and write DDL specific SQL then for better optimisation19:30
jamielennoxbknudson: particularly if i want to use it for something other than RPC, it appears if i'm using it i will understand it already19:31
notmorganwhich case, sure, go for it :)19:31
dstanekthat may be because there's no docs and/or that we always use our own libs in strange ways19:31
mordrednow - if the proposal was also to replace the ORM layer with completely hand-written SQL for all of the things, then I would think it would make more sense, since SQL itself would be a fundamental part of the approach - and in that case, I'd also suggest completely dropping support for sqlite and postgres anyway since nobody uses either19:31
mordrednotmorgan: wow. jinx19:31
notmorganmordred: yep, 100%19:31
notmorganmordred: haha! :) we do occasionally think alike.19:31
jamielennoxbknudson: i am coming to the same conclusion as last time i looked at versionedobjects. oslo = a place for nova to dump out whatever works for them19:33
bknudsonother projects must be using oslo.versionedobjects...19:34
notmorganbknudson: i think cinder is or is moving that way19:34
jamielennoxit's very geared towards RPC, i don't think we'd want it for DB version abstraction19:35
dstanekbknudson: i still don't get how versionedobjects helps here. isn't is an abstraction over object definitions for RPC?19:35
bknudsonmaybe keystone should have a conductor process like nova?19:35
jamielennoxbknudson: no19:36
dstanekbknudson: -2 :-P over architecture for the sake of over architecture19:36
jamielennoxall keystones can talk directly to the DB19:36
jamielennoxthe code we'd have in the conductor can just live in the keystone19:36
jamielennoxanyway, -2 to a conductor, -1 to triggers, -1 to oslo.versionedobjects, +2 to more sleep, back later19:38
dstanekjamielennox: good night19:39
*** su_zhang has joined #openstack-keystone19:41
*** Ephur has quit IRC19:44
*** su_zhang has quit IRC19:46
*** asettle has joined #openstack-keystone19:53
*** markvoelker has joined #openstack-keystone19:57
*** ddieterly is now known as ddieterly[away]19:58
*** asettle has quit IRC19:58
*** woodburn has left #openstack-keystone19:59
*** slberger has quit IRC20:02
*** su_zhang has joined #openstack-keystone20:08
*** edmondsw has quit IRC20:13
*** haplo37__ has joined #openstack-keystone20:14
*** slberger has joined #openstack-keystone20:20
*** ddieterly[away] is now known as ddieterly20:21
*** amakarov has quit IRC20:26
*** hockeynut has joined #openstack-keystone20:27
*** sigmavirus is now known as sigmavirus|away20:29
*** aswadr_ has quit IRC20:31
*** itisha has joined #openstack-keystone20:33
*** gyee has joined #openstack-keystone20:37
*** ChanServ sets mode: +v gyee20:37
*** nkinder has quit IRC20:40
*** chrisshattuck has joined #openstack-keystone20:40
*** slberger has quit IRC20:43
stevemaruhh... what i miss?20:45
stevemarno more triggers? wat.20:45
bretonstevemar: welcome to 2017, where have you been20:46
*** raildo has quit IRC20:46
stevemarbreton: dammit i was gone for a day!20:47
openstackgerritSteve Martinelli proposed openstack/keystone: Add mapping_populate command  https://review.openstack.org/34302820:52
*** markvoelker has quit IRC20:52
*** woodburn has joined #openstack-keystone20:53
*** slberger has joined #openstack-keystone20:56
*** su_zhang has quit IRC20:59
*** ezpz has quit IRC20:59
*** su_zhang_ has joined #openstack-keystone21:01
*** tonytan_brb has quit IRC21:01
*** asettle has joined #openstack-keystone21:02
notmorganstevemar: triggers in SQL are simply a bad idea21:06
notmorganstevemar: you're going to be writing per-engine DDL code and at that point might as well drop the ORM21:06
stevemarnotmorgan: we're only writing per-engine code because sqlalchemry doesn't handle this afaik21:07
notmorganstevemar:right, simply don't do it21:07
stevemarwe could drop sqlite from our trigger support21:07
notmorganstevemar: and also will be different for PGSQL from MySQL21:07
notmorganand galera works ... oddly with triggers21:07
stevemarright, we'll have unit tests for postgres and mysql, the unit tests will test on both now21:08
stevemargranted galera is untested21:08
notmorganthe reason SQL-A doesn't support this is because it's is hard(tm) and engine specific21:08
notmorganmore so than the other DDL specific stuff21:08
notmorganreally, don't do it. see mordred's comments, if there is something you should just trust mordred on, it's SQL things.21:09
notmorgantriggers will result in tears.21:09
stevemarrumor has it he knows some sql stuff n things21:09
notmorganwriting the code in python vs in SQL engine really is going to be more maintainable and more consistent21:10
notmorganas much as it's "write our own code" vs lean on the DB engine to do it21:10
mordredI mean ... I can be convinced if support for not-mysql was totally dropped and aLL database code was migrated to raw SQL21:11
mordredbecaue then you could actually do good database queries and stuff21:11
mordredbut people aren't going to do that21:11
notmorganmordred: i stop short of that because... i mean that kindof is antithetical to the current approach.21:11
mordredso I think sticking the head into the hole halfway in this case is a great way to wind up with multiple pieces of head21:11
mordrednotmorgan: it is21:11
stevemarmordred: what do you recommend then? use versioned objects?21:11
stevemar(boarding soon)21:12
notmorganstevemar: it is the current option used in openstack.21:12
mordredyah - which I believe is what nova is doing with oslo.db right?21:12
notmorganit would be an option21:12
bknudsonmight be better to switch to redis21:12
notmorganbknudson: mongo21:12
notmorganbknudson: document based storage21:12
bknudsonnotmorgan: y, if you want to be webscale.21:12
*** julim has quit IRC21:12
notmorganbknudson: you saw where I was going with this21:12
stevemarbut i'd like to continue the conversation, since it would mean reverting patches21:12
notmorganstevemar: dear god we landed triggers?21:13
stevemarnotmorgan: just some of the stuff surrounding it: https://review.openstack.org/#/q/status:merged+project:openstack/keystone+branch:master+topic:bp/manage-migration21:13
stevemarnew keystone-manage db_sync options21:14
henrynashmodred, notmorgan: I must admit I don't follow the argument of "if you use triggers for a temporary upgrade phase to only copy data between two schemas, then you might as well drop the sue or orm for keystone"21:14
notmorganstevemar: you can still do the work.21:14
notmorganhenrynash: it's triggers themselves are BAD.21:14
notmorganhenrynash: not the concept21:14
notmorganhenrynash: the triggers are going to make life miserable21:14
notmorganhenrynash: and unreliable across engines and a LOT of work.21:14
stevemarnotmorgan: i think we need better evidence other than "its bad" and "life miserable"21:15
notmorganhenrynash: it would be easier to keep the tempoary support code in python.21:15
*** david-lyle has quit IRC21:15
notmorganstevemar: so, the long and the short comes down to this: you are going to write per-DDL code that is very domain specific and almost guaranteed to be untestable in newton in any clustered environment21:16
henrynashnotmorgan: which was strongly argued against by others (dolphm, dstanek, lbragstad)...21:16
*** spzala has quit IRC21:16
notmorganstevemar: in openstack, if it is untested, it is broken21:16
notmorganthis is clear in our history21:16
*** spzala has joined #openstack-keystone21:17
notmorganand this is something that can't be wrong or you destroy openstack deployments21:17
notmorgancode in python is testable without clusters and can be more generic21:17
henrynashnotmorgan: what testing on clustered environments do we do today in opesntack?21:17
*** ddieterly is now known as ddieterly[away]21:17
*** ddieterly[away] is now known as ddieterly21:17
stevemarbut our unit tests now run postgres and mysql now -- and henry added tests21:17
notmorganhenrynash: none, but we don't need to.21:17
stevemar(is coming off as antagonistic, but doesn't mean to)21:18
notmorganhenrynash: since the DB engin is simpl store+replicate21:18
notmorganin DDL specific code you need to increase testing espeically when it comes to "automatic actions"21:18
*** david-lyle has joined #openstack-keystone21:18
notmorganif SQL-A isn't doing it/testing it i worry about our ability to test it/do it21:18
henrynashnotmorgan: why?21:18
notmorganbecause the engine gets weird with replicating these things in general. There is no commonly documented "this doesn't work" in galera, but it's not a widely used case and we are VERY lax in specifying versions of <DB engine> we support21:19
notmorganbasically we treat the DB engine as a simple "store data" box. we do not test beyond that21:20
notmorganthis is extremely worrysome when you start leaning on advanced features (triggers are advanced) in the DB engine21:20
notmorganif SQL-A had an abstraction for it that was more concise and tested, I'd be less worried.21:21
notmorganthis REALLY feels like taking aim at foot and pulling the trigger (pun not intended) hoping that blanks are loaded.21:21
*** spzala has quit IRC21:21
notmorganthe other question is who is writing the DDL specific code?21:21
henrynashnotmorgan: so far me and lbragstad21:22
*** pauloewerton has quit IRC21:22
notmorganlet me step back..21:22
notmorganwhjy are we talking triggers instead of python code?21:22
notmorganthis feels like the whole "cause it's cool" and/or "because we can fire/forget it"21:23
notmorganwhich is IMO not true (either thing)21:23
henrynashnotmorgan: because other cores would not support rolling upgrades with RW support by relying on versions objects21:23
openstackgerritayoung proposed openstack/python-keystoneclient: Update README to include creating a session from a config file.  https://review.openstack.org/35943421:24
notmorganbecause versioned objects suck? or because something else/21:24
henrynashnotmorgan: I have been dragged somewhat kicking (and quietly screaming) into this...having origionally written the whole upgrade process to rely on versioned oonects21:24
mordredI mean, that's how the nova team has been working with the ops community for solving this problem ... I think it would be exceptionally weird for keystone to just make a new weird thing, no?21:25
ayoung(car cdr)21:25
notmorganmordred: thats part of it. i don't think versioned objects are great, but having worked with triggers in MySQL, NDB, MSSQL, and Oracle it only ever seemed to work reliably/well in Oracle and MSSQL21:26
mordredI mean, I'm not a core, but this is solved in openstack, I have no idea why a whole new (and exceedingly brittle) solution would be written from scratch21:26
ayoungWork fine in Postgresql21:26
mordrednotmorgan: yes. exactly21:26
mordredayoung: literally nobody uses postgres with openstack21:26
mordredis has been dropped from the gate for a bunch of projects21:26
notmorganunless you are in very very very clear control of the DB and know all the limitations of the triggers across MySQL/NDB backendsd21:26
notmorganand versions21:26
mordredso we're talking about mysql21:26
notmorganmordred: by literally you mean figuratively, i know 1 deployer does... because he complained on the ML about a bug ;)21:27
ayoungmordred, pretty sure we were just talking DB.  Pretty sure no one runs Openstack on NDB and very few on Oracle or MSSQL either21:27
notmorganmordred:  /me stops snarking about literally21:27
mordrednotmorgan: by literally I mean literally and do not care about statistical outliers21:27
bknudsonwe have a bug saying mysql is 10x slower at something.21:27
notmorganmordred: heheh21:27
henrynashmodred, notmorgan: i have no partiular axe to bear....just that we must have RW rolling upgrade support21:27
mordredayoung: yes. I agree with that21:27
ayoungNever been impressed by MySQL.  Used to have an interview question "Why chose PostgreSQL over MySQL..."21:28
mordredlet's not do THAT conversation21:28
mordredbecause it has literlaly no positive outcome for postgres21:28
henrynashmodred, notmorgan: and since I have now written patches to support both types or RW....just want to see one of them stick!21:28
notmorganayoung: because you have reasons to do so. and there are reasons and uses.21:28
mordredgiven that every super large anything on a database runs on mysql21:28
ayoungthat was years ago.  THe reasons have long since been addressed21:28
notmorganayoung: but *shrug* different choices for different things.21:28
ayoungmordred, um, no.  Every super large anything on a databaseruns either Oracle or DB2.21:29
notmorganhenrynash: i dislike versioned objects, i would generally +2 it with the current architecture of openstack21:29
mordredbut whether or not mysql or postgres are better or worse is irrelevant. the predominant deployment choice for openstakc is by a MASSIVE margin mysql21:29
mordredayoung: not even close21:29
ayoungmordred, yep21:29
mordredayoung: google. facebook. twitter.21:29
ayoungmordred, some day Oravle will die21:29
notmorganhenrynash: i would be inclined to -2 triggers with the current architecture of openstack and complexity introduced with many versions of the DB engines/types/etc21:29
mordredoracle is used for backoffice toys21:29
mordredit is NOT used for LARGE things21:30
mordredthat is 100% dominated by mysql and has been for years21:30
ayoungmordred, may those words become more and more true all the time.  Been burnt by Oracle so many times in the past21:30
notmorganhenrynash: if that helps where I am coming from. in-python code is much much much better suited for our current architecture and lax requirement specifications21:30
*** spedione is now known as spedione|AWAY21:30
notmorganhenrynash: and we can't really be more strict (easily) without breaking whole deployment models of shared resources ooooor drastically increase testing to cover multiuple versions better.21:31
*** hockeynut has quit IRC21:31
notmorganhenrynash: and i fall back to "if it isn't tested, it's broken" which becomes even more prevalent with advanced DB engine features21:31
ayoungKinda not happy with the Triggers approach myself, but was still mulling.  Glad to see it is not just me being a purist21:31
bknudsonone concern I have with the nova approach is that I don't think that they have even done the "cleanup" stage21:33
mordredthen the nova approach should be improved21:33
mordredmaking operators lives harder by introducing a second approach will be welcomed by noone21:33
notmorganhenrynash, stevemar: Bbut that is the level of what i can convey here, and it is based upon talks w/ mordred and anecdoctal experience/personal. so21:33
henrynashnotmorgan: I also prefer not writing DB specific code, but with others -2ing the other approach, we are between a rock and hard place21:34
bknudsondoing the same thing as nova in keystone has been -2?21:34
henrynashbknudson: yes21:34
mordredthat seems very odd21:34
bknudsonwe better watch out we'll get kicked out of openstack.21:35
notmorganhenrynash: this is where the case I make is "the people in the community who grok SQL better than I do (and most everyone else) say this is the wrong approach, please reconsider the -2 on the approach that matches (as distateful as it is) the way other projects are approaching it"21:35
mordrednova has been trailblazing in this area21:35
mordredand I believe the expectatoin from the operators is that other projects will catch up with the work nova has been doing21:35
henrynashmodred: and I origional proposed and we approved a spec around it21:35
mordredI do not believe the operators expect that each project will attempt to solve this from scratch21:35
notmorgani may not like nova's approach. I think it is accepted by the community and operators21:35
mordrednotmorgan: ++21:35
notmorganand fills the needs for our architecture in openstack21:36
henrynashmodred, notmorgan: but others would not support the implementation that this implied21:36
mordredhenrynash: I believe that is/was the right thing to do21:36
notmorganmordred: this sounds like something the TC needs to really mandate a direction on.21:36
notmorgan"rolling updates will conform roughly to X,Y,Z"21:37
notmorgannot mandating rolling updates are needed, just "use oslo" type mandate21:37
notmorgan(ignoring swift, obv)21:37
mordrednotmorgan: well, honestly, it sounds like communication/education is sorely needed21:37
notmorganmordred: probably.21:37
mordredbecause I find it hard to believe that keystone cores would -2 such an approach when presented with the informatoin that the approach has bene developed hand in hand with our operator community over the last 2 years21:38
mordredso I can only imagine that they are not aware of that21:38
notmorganbut the place i'd start is the above quoted thing... basically leaning to the folks who know SQL and how it works under the hood + oepnstacks' architecture say triggers are not a good idea (for a number of reasons)21:39
*** su_zhang_ has quit IRC21:42
*** su_zhang has joined #openstack-keystone21:42
*** sdake has quit IRC21:45
*** asettle has quit IRC21:45
*** sdake has joined #openstack-keystone21:45
*** sdake has quit IRC21:45
bknudsonI wonder what the reason was for the rejection of the nova approach? Because we didn't want oslo.versionedobjects?21:46
*** sdake has joined #openstack-keystone21:46
bknudsonMaybe oslo.versionedobjects isn't a requirement.21:46
notmorganbknudson: ++21:50
notmorganbknudson: both things good to know21:50
*** chrisshattuck has quit IRC21:51
*** spzala has joined #openstack-keystone21:53
Gorian|workwow! People talk in here!21:56
*** atod has joined #openstack-keystone21:57
*** ddieterly is now known as ddieterly[away]21:57
*** tonytan4ever has joined #openstack-keystone22:02
notmorganGorian|work: we got kicked out of openstack-dev at one point because we talk too much :P22:04
Gorian|workhaha, wow22:04
Gorian|workI hardly ever see activity in the OpenStack channels22:04
notmorganGorian|work: -nova, -infra, -cinder, and -keystone i know get decent chatter22:05
notmorganand -meeting (obviously)22:05
notmorganchannel that discusses all the things related to the openstack infrastructure22:06
notmorgangerrit, zuul, nodepool, etc22:06
notmorgan(operating it for penstack)22:06
Gorian|workoh cool22:06
notmorgan#openstack-infra (that is)22:06
notmorganthe -<name> is just shorthand :)22:06
*** tonytan4ever has quit IRC22:07
Gorian|workyeah, I gathered that :P22:07
Gorian|workwho are you by the way? Since you aren't morgan22:07
Gorian|workyou can't just identify yourself by who you aren't!22:07
rodrigodsstevemar, around?22:07
notmorgani am also morgan22:07
Gorian|workwhat? How does that work?!22:08
notmorgani camp on both names ;)22:09
notmorganbut irc is being cranky and not letting me change nicks22:09
*** ddieterly[away] is now known as ddieterly22:09
*** notmorgan is now known as morganfainberg22:09
Gorian|workawww, that sucks22:09
*** morganfainberg is now known as morgan22:09
morganthere we go22:10
*** slberger has left #openstack-keystone22:10
morgansomeone tries to login with my account because it's "morgan" about 200 times a day22:10
morgandue to a badly configured thunderbird client22:10
morganbut it is mine muhahahahaha22:11
* morgan laughs evilly22:11
*** morgan is now known as notmorgan22:11
* notmorgan cackles manaiacally22:12
*** gagehugo has quit IRC22:20
*** ayoung has quit IRC22:27
*** ddieterly is now known as ddieterly[away]22:31
*** baffle has quit IRC22:34
*** michauds has quit IRC22:36
*** ddieterly[away] has quit IRC22:36
*** shaleh has quit IRC22:41
*** sdake has quit IRC22:44
*** sdake has joined #openstack-keystone22:44
*** spzala has quit IRC22:48
*** baffle has joined #openstack-keystone22:48
*** BjoernT has quit IRC22:49
*** spzala has joined #openstack-keystone23:00
*** chlong has quit IRC23:03
*** tonytan4ever has joined #openstack-keystone23:03
*** spzala has quit IRC23:05
*** tonytan4ever has quit IRC23:08
*** iurygregory_ has joined #openstack-keystone23:08
*** ddieterly has joined #openstack-keystone23:10
*** ddieterly is now known as ddieterly[away]23:11
*** ddieterly[away] is now known as ddieterly23:13
*** esp has joined #openstack-keystone23:14
notmorganyay i have real internet again23:17
*** erhudy has quit IRC23:22
*** bigdogstl has joined #openstack-keystone23:25
*** Gorian|work has quit IRC23:25
*** bigdogstl has quit IRC23:29
*** spzala has joined #openstack-keystone23:34
*** spzala has quit IRC23:39
*** eandersson_ has joined #openstack-keystone23:40
*** eandersson has quit IRC23:43
*** bigdogstl has joined #openstack-keystone23:45
*** bigdogstl has quit IRC23:50
*** spzala has joined #openstack-keystone23:56
*** cheran has joined #openstack-keystone23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!