18:01:46 <heckj> #startmeeting 18:01:47 <openstack> Meeting started Tue May 8 18:01:46 2012 UTC. The chair is heckj. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:48 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:59 <heckj> Not much on the topic list for this week 18:02:25 <heckj> Next week we've got rafaduran wanting to discuss https://bugs.launchpad.net/keystone/+bug/963098 and a related blueprint that I think he's creating 18:02:27 <uvirtbot> Launchpad bug 963098 in keystone "Keystone isn't acting on consecutive failed logins" [High,Triaged] 18:02:33 <heckj> #topic open discussion 18:02:50 <gyee> heckj, I need some love on the serviceId review 18:02:59 <rafaduran> the bluprint https://blueprints.launchpad.net/keystone/+spec/improve-keystone-security 18:03:11 <annegentle> heckj: I have a question re: keystone database support also 18:03:21 <heckj> gyee: could you put a link up here for folks to review? 18:03:24 <dolphm> v.next api draft link- now, tomorrow, soon? 18:03:25 <rafaduran> and a draft https://review.openstack.org/#/c/7239/ 18:03:38 <gyee> https://review.openstack.org/#/c/7010/ 18:03:47 <heckj> dolphm: soon - haven't made as much progress last week and weekend as I'd hoped 18:04:15 <heckj> annegentle: question? 18:04:32 <heckj> #link https://review.openstack.org/#/c/7010/ <-- requesting review 18:04:47 <annegentle> heckj: is postgresql tested at all as a backend for the catalog? I'm trying to find the bug where dolph said he fixed it, still looking 18:04:52 <heckj> #link https://blueprints.launchpad.net/keystone/+spec/improve-keystone-security <-- for discussion next week 18:04:59 <annegentle> heckj: I wondered if it was going to be punted to openstack-common 18:05:08 <heckj> annegentle: not explicitly testing postgresql 18:05:29 <dolphm> annegentle: https://bugs.launchpad.net/keystone/+bug/987121 ? 18:05:30 <uvirtbot> Launchpad bug 987121 in keystone "strict constraint for database table creation" [Medium,Fix committed] 18:05:37 <annegentle> ok, it's https://bugs.launchpad.net/keystone/+bug/885426 18:05:38 <uvirtbot> Launchpad bug 885426 in keystone "type error with postgresql" [Medium,Fix released] 18:05:53 <annegentle> apparently one of the CSS OSS guys tested it for the manual they just released, and said there's still a problem. 18:06:30 <dolphm> annegentle: tested essex-3? 18:06:50 <annegentle> dolphm: no, their manual documents what shipped with 12.04 18:07:03 <dolphm> annegentle: that bug fix isn't in the current codebase (pre-redux) 18:07:27 <dolphm> annegentle: they must be seeing a new bug 18:07:27 <annegentle> he didn't know whether to open a new bug or reopen that old one 18:07:31 <annegentle> ok 18:07:52 <ayoung_> rafaduran, let me look 18:07:56 <heckj> dolph - should we reset that to triaged/confirmed and hit it now? 18:08:02 <heckj> dolphm: ^^ 18:08:35 <dolphm> heckj: that bug is unreproducable against ksl 18:09:02 <heckj> Ah - 18:09:17 <heckj> annegentle: if it's still an issue, let's open a new bug with as much repro info as we can against it 18:09:32 <annegentle> I'll have him open a new bug against keystone then. He also was unsure if it was a distro problem. 18:09:52 <ayoung> annegentle, I can give it a test on Fedora or RHEL if that will help 18:10:10 <rafaduran> ayoung_:I'm sorry, bu I'm not really sure what are you asking for 18:10:13 <annegentle> ayoung: sure, that would help. I want to genericize the install/deploy guide for those distros too 18:10:20 <ayoung> rafaduran, disregard... 18:10:36 <ayoung> annegentle, that is the Grail, isn't it 18:11:19 <ayoung> we were discussing (internal IRC) the multiple ways people automate install 18:11:51 <ayoung> annegentle, CC me when you open the ticket 18:12:00 <annegentle> ayoung: you got it 18:12:46 <ayoung> rafaduran, for the "3 times and you are "(locked) out" issue, we were discussing whether this is a generic Keystone problem or should it just be for the Keystone Database impl 18:13:18 <ayoung> It seems like many people that have their own Identity Managment would get account locking from a centralized source already...ie LDAP 18:13:32 <gyee> :) 18:13:49 <gyee> especially AD, I had my locked the other day 18:14:11 <ayoung> Sorry gyy, thought i was better at hacking your account. I'll get it right next time :) 18:15:25 <gyee> but keystone password policy might be a nice to have though 18:15:29 <ayoung> rafaduran, but regarding the general "imporve security" I suspect we should focus on reusing what has been done for HTTPD 18:15:43 <ayoung> as opposed to trying to bolt things on to eventlet 18:15:54 <ayoung> which leads to my next topic... 18:16:03 <rafaduran> ayoung: but reporting and rate limit doesn't conflict that 18:16:17 <ayoung> rafaduran, agreed that reporting stands alone. 18:16:21 <ayoung> Not sure about rate limit 18:16:27 <ayoung> you might be right there. 18:16:55 <ayoung> There also might be something in HTTPD to perform that as well. 18:17:03 <liemmn> ayoung, is the HTTPD SSL stuff ready for review yet? We are looking to push the 2-way SSL back in... 18:17:19 <ayoung> liemmn, SSL or Client cert? 18:17:26 <ayoung> HTTPD SSL works 18:17:27 <liemmn> 2-way SSL 18:17:31 <liemmn> client cert 18:17:43 <ayoung> liemmn, OK, so that Requires HTTPD 18:17:47 <liemmn> yep 18:18:01 <ayoung> and so I was asking if we can make that the one and only, or if we need to keep eventlet around? 18:18:15 <ayoung> heckj, ^^ is really a question for you 18:18:31 <heckj> sorry - distracting. reading 18:18:46 <ayoung> liemmn, however, I have a paste I want to share which is the general approach 18:19:07 <liemmn> sure... please shoot it my way and I can take a look... thx 18:19:16 <ayoung> http://fpaste.org/9PLL/ 18:19:29 <ayoung> lots of duplicated code there from higher, but the gist is this 18:19:41 <ayoung> configure HTTPD to do authentication for you, 18:19:48 <ayoung> and then in Keystone, the rule is 18:19:58 <ayoung> 1. Look for UserId/ Password in the message itself 18:20:04 <ayoung> 2. Look for a token 18:20:09 <ayoung> 3. Look for REMOTE_USER 18:20:20 <ayoung> REMOTE_USER means that HTTPD authenticated you already 18:20:31 <heckj> ayoung: we should keep both around from an internal code point of view. 18:21:41 <heckj> ayoung: we could also really use some documentation in the ReST files (doc/source) walking someone through how to configure to take advantage of the 2-way SSL and such. 18:22:09 <ayoung> heckj, for 2 Way SSL , I'm not there yet, but I would be happy to once I get it working. 18:22:19 <gyee> ayoung, that's for authentication only right? what about token validation? 18:22:31 <liemmn> From the middleware perspective, there is still a need to have both 2-way SSL and normal token validation... i.e., I only allow these hosts with these signed certs to do valiidate token. 18:22:34 <ayoung> It is going to be slightly different for Fedora and Debian based distros due to the way that HTTPD gets set up. We already see that in Devstack 18:22:46 <ayoung> gyee, same general rule 18:22:55 <ayoung> except the user part doesn;t apply 18:23:08 <ayoung> so only look for admin auth token and then REMOTE_USER 18:23:16 <ayoung> and then confuirm that remote user has admin privs 18:24:10 <gyee> heckj, yeah, lots of docs for ayoung 18:24:15 <ayoung> heckj, how about for a devstack install? Is it OK if we go HTTPD there? 18:25:15 <heckj> ayoung: it should be an option for the devstack setup as well, but I don't see anything wrong with the idea. 18:25:35 <ayoung> heckj, I see Eventlet being troublesome 18:25:41 <ayoung> also 18:25:47 <ayoung> Devstack is getting pretty complex 18:25:59 <ayoung> and I would like to avoid putting more knobs to turn in there 18:26:08 <ayoung> It comes down to most people using the default options 18:26:18 <ayoung> except maybe for the piece they are working on. 18:26:28 <ayoung> So I'd like, in devstack, for httpd to be the default 18:26:36 <rafaduran> ayoung: why do you think eventlet is a problem? 18:27:07 <heckj> ayoung: that's the right place to start, but I'm not sure I agree that it *should* be default when I haven't seen it working yet. I don't doubt your work, just want to see it operational before we make it a default 18:27:17 <ayoung> rafaduran, 1. SSL support is spotty in Python, not just Eventlet. 2. IPv6, 3.Client Certs and other auth versions. 18:27:20 <ayoung> heckj, understood 18:27:26 <ayoung> but 18:27:35 <ayoung> if I am making it work as a devstack patch 18:27:43 <ayoung> I write it one way if it is going to be the one true approach 18:27:54 <ayoung> and another way if it is going to be just an option 18:28:34 <ayoung> rafaduran, rafaduran there are also issues in the Eventlet_>SQL code that we are seeing else where that I fear are going to bite us 18:29:00 <liemmn> IMHO, SSL should be an option... For someone who wants to get started quickly with Keystone, certificates is cumbersome. Make testing more cumbersome too. 18:29:12 <ayoung> liemmn, that is correct 18:29:26 <ayoung> certs are only an option if the site admin sets them up 18:29:52 <ayoung> the nice thing about fronting with HTTPD is that they can even use basic-auth without changing the Python side of things 18:30:06 <ayoung> the changes are confined to /etc/httpd/conf.d/keystone.conf 18:30:10 <ayoung> (on Fedora) 18:30:29 <ayoung> liemmn, but, if you want to do , say Kerberos (AD) you get that, too 18:31:26 <liemmn> yeah... reading your blog on setup howto :) 18:32:08 <liemmn> are we unit testing with httpd, too? 18:33:01 <ayoung> liemmn, good question...I would argue that if you are running a webserver, you are probably not "Unit testing" but that is neither here nor there 18:33:10 <gyee> ayoung, for certificate authentication, how do you map user certificate to keystone user ID? 18:33:45 <ayoung> gyee, I was thinking that REMOTE_USER should be username. So it is probably the Principal in the X509 18:34:10 <gyee> that configurable in your implementation? 18:34:14 <ayoung> gyee, I don't think we want to put the UserID into the Certificates, do you? 18:34:47 <ayoung> gyee, no, but Client Certs are not done or tested yet anyway. 18:34:50 <liemmn> user name sounds good to me 18:35:03 <ayoung> Iwas focusing on getting Keystone to work with Nova first. 18:35:12 <ayoung> I have it working with Glance. 18:35:32 <gyee> I am looking at http://fpaste.org/9PLL/ 18:35:46 <gyee> how does httpd translate user cert into user_ref? 18:36:16 <liemmn> Line 12 18:36:25 <ayoung> gyee, just to set expectations corectly: I just wrote but have not tested that code...it was more a "thinking along these lines" 18:36:50 <ayoung> self.identity_api.get_user_by_name(context=context, user_name=context['REMOTE_USER']) 18:36:58 <rafaduran> ayoung: a user can be authenticated for a given tenant too? 18:37:08 <gyee> oh ok, I'll wait for your rst doc then 18:37:15 <liemmn> I do think we want to test 2-way SSL with unit tests... I am a big fan of unit tests... We were doing it before; we should be able to do it now. 18:37:33 <gyee> +2 18:37:35 <ayoung> liemmn, agreed. The question, then, is how to run HTTPD for unit tests 18:37:58 <liemmn> I have no answer yet, but... just something to keep in mind when it comes to configuration :) 18:39:19 <ayoung> I'd guess something along the lines of : "see if you can run HTTPD listening on 5000 or 35757 as the current user, reading all config info out of the git tree:" 18:40:12 <ayoung> liemmn, but I'd almost think that Eventlet+basic_auth would be a good first step 18:40:32 <ayoung> assuming that Eventlet then sets REMOTE_USER, the rest of the Python code would remain unchanged 18:43:18 <liemmn> The client needs to validate server cert as well... so, need HTTPD there. 18:43:24 <ayoung> liemmn, so Eventlet , SSL and Client certs should really be a sn upstream Eventlet feature, not anything specific to any Openstack 18:43:57 <ayoung> liemmn, is that really unit testing Keystone, or just that SSL is set up? 18:44:00 <liemmn> yeah, actually, if my memory serves me correctly, that's where I made the changes... 18:44:20 <ayoung> I mean, you can always run curl -k ... 18:45:00 <ayoung> liemmn, OK, so there should be a pretty simple way to run and test SSL in Eventlet as well as HTTP, if you can find your notes, send them to me 18:45:35 <heckj> liemmn: would love to see the same ^^ if you're finding them 18:45:37 <gyee> he's code were on the E3 branch I think 18:46:19 <ayoung> cc2330a8e1c1d55e6ae23d05ab5d09d3fd511ea7 18:46:19 <gyee> before the KSL cut over 18:46:24 <liemmn> (digging up old mails... :) ) 18:46:39 <liemmn> https://review.openstack.org/#/c/1038/ 18:47:37 <heckj> #link https://review.openstack.org/#/c/1038/ <-- two way SSL from prior to KSL integration 18:47:39 <liemmn> (that was my very first commit in Openstack... so, I f'up a lot... laugh it up :) ) 18:47:49 <liemmn> yeah 18:48:06 <ayoung> 126 sslsocket = eventlet.wrap_ssl(socket, certfile=certfile, 18:48:07 <ayoung> 127 keyfile=keyfile, 18:48:07 <ayoung> 128 server_side=True, cert_reqs=cert_reqs, 18:48:07 <ayoung> 129 ca_certs=ca_certs) 18:48:16 <ayoung> that seems to be the heart of it... 18:48:21 <ayoung> https://review.openstack.org/#/c/1038/9/keystone/common/wsgi.py 18:48:21 <liemmn> yep 18:49:12 <liemmn> should be a fairly small effort to get it working with eventlet 18:50:36 <ayoung> heckj, I keep hearing that SSL support in Python is problematic. Buy oviously Swift has been running this way for years...what am I missing? Do real world Swift deployments just run with Hardware SSL ? 18:51:11 <notmyname> ayoung: you should run swift with external ssl termination (in the load balancer or something like that) 18:51:18 <heckj> ayoung: I'm not sure what's behind the "SSL in python is problematic" - I haven't done much with it personally to know what the issues are or have been. 18:51:55 <gyee> python httplib2 does not do server cert validation 18:52:09 <ayoung> notmyname, doesn't that defeat the purpose of using eventlet or a similarly event driven web server? 18:52:54 <notmyname> ayoung: https://github.com/notmyname/ssl_eventlet_slowloris 18:53:27 <ayoung> heckj, IIUC it comes down to Python taking the GIL when doing the SSL, which means that a web server blocks for each request, but I am not sure if that is the whole story 18:53:34 <notmyname> ayoung: it's not as much a problem with eventlet as much as how python exposes the socket to eventlet. but I'd like to explore more on this (it's near the bottom of my todo list) 18:54:46 <ayoung> notmyname, would swift benefit from HTTPD support? 18:55:19 <ayoung> we can have that conversation later...times almost up for this meeting and I've waxed poetic 18:55:27 <heckj> heh 18:55:30 <heckj> 5 minutes left 18:57:11 <gyee> heckj, I've started the domain bp impl 18:57:16 <heckj> anything last minute before I close this down? 18:57:22 <heckj> gyee: excellent! 18:57:24 <gyee> should I stash the stuff in contrib or identity? 18:57:46 <gyee> in the absence in /v3.0 18:57:53 <heckj> start with contrib/ - and we'll work on moving/merging when we get /v3 settled 18:57:59 <gyee> cool 18:58:11 <liemmn> quick question... heckj, how are the v3 api comming? 18:58:23 <liemmn> any etherpad? 18:58:40 <heckj> leimmn: way back to the begining of the meeting - didn't get the time I wanted this past week & weekend to work on it. 18:58:44 <heckj> No etherpad 18:59:07 <heckj> will be in google docs for feedback - etherpad is just a touch too unstructured for what I want 18:59:21 <liemmn> cool... please keep me and gyee in the loop.... thx 18:59:26 <heckj> absolutely 18:59:32 <heckj> Okay - that's it for today! 18:59:35 <heckj> #endmeeting