Wednesday, 2015-04-08

*** dave-mccowan has joined #openstack-barbican00:13
*** alee has quit IRC00:24
*** SheenaG1 has quit IRC00:29
*** alee has joined #openstack-barbican00:36
*** everjeje has quit IRC00:36
*** alee has quit IRC01:00
*** alee has joined #openstack-barbican01:13
*** alee has quit IRC01:29
*** alee has joined #openstack-barbican01:31
*** kebray has joined #openstack-barbican01:34
*** alee has quit IRC01:35
*** alee has joined #openstack-barbican01:48
*** jvrbanac has quit IRC02:00
*** zz_dimtruck has quit IRC02:00
*** jillysciarilly has quit IRC02:00
*** alee has quit IRC02:00
*** eglute has quit IRC02:00
*** hockeynut has quit IRC02:01
*** lbragstad has quit IRC02:01
*** tdink has quit IRC02:02
*** jroll has quit IRC02:02
*** zz_dimtruck has joined #openstack-barbican02:07
*** jvrbanac has joined #openstack-barbican02:07
*** jillysciarilly has joined #openstack-barbican02:07
*** zz_dimtruck is now known as dimtruck02:07
*** hockeynut has joined #openstack-barbican02:07
*** jroll has joined #openstack-barbican02:07
*** eglute has joined #openstack-barbican02:07
*** tdink has joined #openstack-barbican02:08
*** jroll has quit IRC02:08
*** jroll has joined #openstack-barbican02:08
*** lbragstad has joined #openstack-barbican02:11
*** alee has joined #openstack-barbican02:13
*** alee has quit IRC02:21
*** SheenaG has joined #openstack-barbican02:28
*** alee has joined #openstack-barbican02:33
*** alee has quit IRC02:39
*** alee has joined #openstack-barbican02:52
*** alee has quit IRC02:56
*** alee has joined #openstack-barbican03:09
*** alee has quit IRC03:13
openstackgerritDave McCowan proposed openstack/barbican: Add Functional Tests for Private Key Secret Type
*** xaeth_afk is now known as xaeth03:21
*** SheenaG has quit IRC03:31
openstackgerritDave McCowan proposed openstack/barbican: Add Functional Tests for Private Key Secret Type
*** tkelsey has joined #openstack-barbican03:49
*** tkelsey has quit IRC03:55
*** rm_you| has joined #openstack-barbican04:02
*** rm_you has quit IRC04:05
*** SheenaG has joined #openstack-barbican04:10
*** xaeth is now known as xaeth_afk04:11
*** woodster_ has quit IRC04:20
*** woodster_ has joined #openstack-barbican04:46
openstackgerritJohn Vrbanac proposed openstack/python-barbicanclient: Fixing the broken functional tests
*** dave-mccowan has quit IRC05:02
*** SheenaG has quit IRC05:04
*** gyee has quit IRC05:23
*** alee has joined #openstack-barbican05:37
openstackgerritOpenStack Proposal Bot proposed openstack/barbican: Imported Translations from Transifex
*** tkelsey has joined #openstack-barbican06:42
*** tkelsey has quit IRC06:46
*** jaosorior has joined #openstack-barbican06:56
openstackgerritDouglas Mendizábal proposed openstack/barbican: Document Symmetric Secret Type
*** everjeje has joined #openstack-barbican07:34
*** kebray has quit IRC07:39
*** woodster_ has quit IRC07:50
*** nickrmc83 has quit IRC09:28
*** nickrmc83 has joined #openstack-barbican09:30
openstackgerritEverardo Padilla Saca proposed openstack/barbican: Remove str() casting for the client_message variable
*** rm_work has quit IRC10:04
*** rm_work has joined #openstack-barbican10:06
*** rm_work has joined #openstack-barbican10:06
openstackgerritThomas Herve proposed openstack/barbican: Return container not found before ACL checks
*** nickrmc83 has quit IRC10:15
*** nickrmc83 has joined #openstack-barbican10:15
*** darrenmoffat has quit IRC10:21
*** darrenmoffat has joined #openstack-barbican10:22
openstackgerritThomas Herve proposed openstack/barbican: Return container not found before ACL checks
*** woodster_ has joined #openstack-barbican12:14
*** rellerreller has joined #openstack-barbican12:45
*** dave-mccowan has joined #openstack-barbican12:47
*** nkinder has quit IRC13:12
*** alee has quit IRC13:42
*** alee has joined #openstack-barbican13:46
aleewoodster_, rellerreller , redrobot  jaosorior  - please take a look at
jaosorioralee: sure13:49
jaosoriorthough it seems to be failing13:49
jaosoriorlets see why13:49
aleewoodster_, rellerreller , redrobot , jaosorior - not sure why its not passing the gate -- appears to pass locally for me and for dave (so its not an openstack issue).  Seems like every time I run the recheck, a different couple of tests fail.13:49
aleeI think some things are timing out on the gate.13:49
aleeif you can, please run the tests locally and see what happens.13:50
jaosorioralee: does it tend to fail in the same tests?13:50
aleethere is one test in particular that seems to fail consistently -- api.v1.functional.test_certificate_orders.CertificatesTestCase.test_create_stored_key_order13:51
aleebut I dont know why ..13:51
aleerequests.exceptions.ConnectionError: HTTPConnectionPool(host='', port=9311): Max retries exceeded with url: /v1/orders (Caused by ProtocolError('Connection aborted.', BadStatusLine("''",)))13:51
aleeother tests fail randomly with the same reason13:52
*** xaeth_afk is now known as xaeth13:53
aleejaosorior, woodster_ - aha!  I think I found it !13:58
jaosorioralee: what's up?13:58
*** chlong has quit IRC13:58
aleeDataError: (DataError) (1406, "Data too long for column 'value' at row 1"13:59
aleeyup - trying to save the generated csr13:59
jaosoriorwhat database backend is being used there?13:59
aleejaosorior, how do I find that out?  isn't the problem in the table column size definition?14:00
jaosoriormight be14:01
jaosoriorbut if you say it works for you14:01
jaosoriorit might also be database dependant14:01
jaosorioractually the tests just finished running on my machine14:02
jaosoriorand it fails here ERROR: functionaltests.api.v1.functional.test_certificate_orders.CertificatesTestCase.test_create_stored_key_order14:02
*** nkinder has joined #openstack-barbican14:02
jaosoriorIn the same way14:02
jaosoriorrequests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))14:02
aleejaosorior, not sure why it works for me -- maybe openssl for my version generates a smaller csr14:03
aleeand we're just hitting the limit14:03
jaosoriorcould be14:03
aleejaosorior, do you see the same error in your barbican logs?14:03
jaosoriorI actually have a newer version of OpenSSL (using 1.0.2a)14:04
*** paul_glass has joined #openstack-barbican14:04
aleejaosorior, OrderBarbicanMetadatum has -->14:05
aleekey = sa.Column(sa.String(255), nullable=False)14:05
alee    value = sa.Column(sa.String(255), nullable=False)14:05
jaosorioralee: re-running the tests, had to purge my logs since there's too much stuff14:05
jaosoriorIs there a limit to how big CSRs can be?14:06
aleesame for OrderPluginMetadatum14:06
rellerrelleralee I can take a look at the patch, but it will have to wait until this afternoon. Lot's of fires here this morning.14:07
jaosorioralee: I checked now, and I get the exact same log14:07
aleewell I guess it depends on the key size14:07
aleerellerreller, thanks14:07
jaosoriormaybe we'll need to compress the CSRs? or change that value14:07
jaosoriorwell, that size14:08
aleejaosorior, yeah -- need some input from db people here -- woodster_ ?14:12
aleejaosorior, woodster_, rellerreller  - looks like all the metadatum fields (secret, order, barbican metadata) have the same 255 column length.14:12
aleejaosorior, to make the column unbounded - would you use sa.Text instead?14:13
jaosorioralee: I guess, though I'm not sure if we want to make that unbounded14:14
jaosorioryeah, indeed it would be sa.Text14:15
aleecertificateAuthorityMetadatum is also 255 --> will need to expand that one to be able to accomodate ca signing certs and intermediate chains14:16
aleejaosorior, woodster_ - at the very least, it should be ok to make OrderPluginMetaDatum, OrderBarbicanMetadatum and CertificateAuthorityMetadatum unbounded - or with a very high bound.  The data that goes in there either comes from a plugin or from the barbican server itself.14:20
jaosorioralee: my concern is the possibility of an attacker taking advantage of this and providing very big payloads for the metadatum info. But then again that my only be tinfoil hat thinking.... and it could also be that we are in a way protected by the boundaries of the packages that can be sent to Barbican, I guess at some point we would return 413?14:22
aleeactually the same is true of SecretMetadatum -- those inputs come from the plugin too.14:22
aleejaosorior, right -- the attacks would have to come from the plugin side14:23
alee(not from the front facing interface14:24
aleewoodster_, redrobot ??14:24
jaosoriorbut some of the metadatum would actually be coming from the front facing interface14:24
aleejaosorior, no14:24
aleejaosorior, the data for an order for example is stored in the meta field of the order table14:25
aleewhich is a field that contains a json blob14:25
alee(not sure if its bounded or not)14:25
jaosorioryeah, but that's for the order, what about the secretmetadatum?14:26
aleethe order has two metadata fields associated with it -- orderplugin and barbicanmeta -- the first is used by the plugins to store data abiout the order, the second by barbican itself to store data.14:26
aleefor secret metadata -- ...14:27
aleejaosorior, I think that is only used by the plugin to store stuff about the secret14:27
jaosorioruhm, then you're right14:27
jaosoriorit would be up to the plugin14:27
alee(for example an id)14:27
*** rellerreller has quit IRC14:28
aleejaosorior, guess we'll wait to hear from woodster_ and redrobot as to how they would like to proceed.  you can still review the CR though :)14:29
*** xaeth is now known as xaeth_afk14:51
* jvrbanac is wondering if the client devstack gate is somehow using an older client14:59
*** kebray has joined #openstack-barbican15:00
*** xaeth_afk is now known as xaeth15:13
jaosoriorjvrbanac: how come?15:15
*** kebray has quit IRC15:16
jvrbanacjaosorior, my changes fixes the very thing causing the functional test errors15:25
woodster_alee, jaosorior just catching up...we do put an upper bound on the overall input request (10k by default I believe)15:25
woodster_well, 10k for the secret payload. I think the overall bound is higher than that15:25
jaosoriorjvrbanac: that's weird...15:29
jaosoriorwoodster_ do you  have an opinion about getting the metadatums to be unbounded in the database?15:29
woodster_a concern raised by our dbas here was with clobs that are frequently retrieved but not used, due to perf impact.15:30
aleewoodster_, which clobs would those be?15:31
woodster_another option could be to have tables just for this specialized large scale data?15:32
woodster_sorry, sa.Text is essentially a clob15:32
aleewoodster_, right, I guess I'm wondering under which context the clob would not be used15:32
woodster_well, for a lot of the key/value info on those metadata data tables the 255 limit is fine. It is just things like certs and so forth that break out of that. So maybe those could be on normalized tables with FKs to orders/secrets/containers and what not15:33
*** SheenaG has joined #openstack-barbican15:36
aleewoodster_, so now we want extra tables for large values? so to process an order, I need to load up order-plugin-meta, barbican-meta, barbican-large-meta and order-plugin-large-meta?15:37
jaosorioralee: I'm not very keen on that option15:39
aleeme neither15:39
woodster_alee, well instead of barbican-large-generic-anything-goes-meta, I'm saying something like order-certificates or container-certificates and so specific tables that are only queried from if needed during a specific workflow. So generating an AES key (for example) would not incur the cost of pulling in generic clob data.15:39
aleewoodster_, another possibility is to use a separate table for large meta, but make it transparent.  So, when storing large meta, the plugin (or server) would need to call store_large_meta() and it will end up in the clob table.  when retrieving though, it will just get the value.15:43
aleenot sure how to do that but essentially some values in the meta table will point to the large meta table15:44
aleethat way the code/plugin is explictly specifying which values are expected to be large.15:44
woodster_alee, yeah, well my concern is with a lot of generic data...I much prefer specific attributes (so a specific field in the response dto from the plugin that barbican saves) over generic blobby stuff. Just my thoughts though :)15:45 if a specific attribute is unbounded, I can dedicate a table to it, roughly speaking15:45
aleewoodster_, the whole idea behind the plugin meta table was to not constrain the plugin -- the plugin can store whatever it needs.15:46
aleewoodster_, so if I understand your concern - performance on clobs is much slower than normal columns - even when the data is small?15:48
aleewoodster_, ?16:01
aleewoodster_, I'm not a db guy, so I don't understand the performance implications.  The immediate problem is that we have at least bits of data that will exceed the 255 maximum -- a generated_csr stored in the barbican_meta_table and signing certs and intermediate certs which are in the CA metadata table.16:05
aleewe need a way to make sure we can store those.16:05
woodster_alee, the perf impact is basically it. I can try to get a better answer on what that impact is from our dbas though.16:05
woodster_alee, certainly to fix the immediate issue we can go with clobs and then revisit for liberty16:06
aleewoodster_, ok -- and we can do that purely for the CA meta and barbican meta tables if you like -- although I'd suggest for all the meta tables16:07
aleeie. secret meta and order-plugin-meta as well.16:07
aleebut to solve the immediate problem, we just need barbican-plugin-meta and ca-meta16:08
*** dave-mccowan has quit IRC16:09
aleewoodster_, so ca meta and barbican meta only? or secret_meta and order_plugin_meta as well?16:10
openstackgerritJohn Vrbanac proposed openstack/python-barbicanclient: Fixing the broken functional tests
woodster_alee, maybe start with the minimum we need for now?16:11
aleewoodster_, sure - we can revisit as we need to16:11
*** dave-mccowan has joined #openstack-barbican16:13
*** kebray has joined #openstack-barbican16:17
*** gyee has joined #openstack-barbican16:20
*** jkf has joined #openstack-barbican16:31
*** SheenaG has left #openstack-barbican16:54
*** SheenaG has joined #openstack-barbican16:54
*** xaeth is now known as xaeth_afk16:55
jvrbanacredrobot, same problem16:57
openstackgerritJohn Vrbanac proposed openstack/python-barbicanclient: Fixing the broken functional tests
openstackgerritMerged openstack/barbican: Remove str() casting for the client_message variable
*** rellerreller has joined #openstack-barbican17:48
*** jaosorior has quit IRC17:52
*** everjeje has quit IRC17:56
*** kfarr has joined #openstack-barbican17:57
*** igueths has joined #openstack-barbican18:04
jvrbanacredrobot, down to 2 failures now:
jvrbanacredrobot, any thoughts on why those requests would be coming back with a 40318:17
jvrbanacwoodster_, ^18:17
jvrbanacredrobot, woodster_,
jvrbanacredrobot, woodster_, is there a different policy for devstack than local configuration?18:20
*** nkinder has quit IRC18:34
jvrbanacIt doesn't look like it...18:44
*** dave-mccowan has quit IRC18:47
jvrbanacAnyone have any ideas??18:55
rm_workI always see this, so I assume it's a red herring: INFO barbican.openstack.common.policy [-] Can not find policy directory: policy.d18:57
rm_workNever been sure what causes that18:57
jvrbanacrm_work, not that one. This is where the barbican client devstack gate is failing a couple tests with 403s that give the following traceback in Barbican
rm_workthat's what i'm saying -- the policy.d directory thing is a red herring, right?18:59
rm_workI assume unrelated, because i see it ALL the time, and if that were the problem i assume every test would fail, not just a couple19:00
jvrbanacthe directory thing yes. But it's it's talking about our rbac roll not being correct or something19:00
rm_workyeah, the rest I don19:00
jvrbanacLocally it returns a 40419:00
rm_work*don't know, i don't see that in my devstack either19:00
rm_workah is this a new test?19:00
rm_worklet me try in my env19:00
rm_workgotta re-spin, sec19:00
openstackgerritAdam Harwell proposed openstack/barbican: Use the new Devstack external plugin method
jvrbanacold tests. I'm trying to fix the barbican-client gate19:01
rm_work^^ BTW when this passes checks, it should be good to merge19:01
jvrbanacrm_work, grab this CR:
*** paul_glass has quit IRC19:01
rm_workoh, client19:01
rm_workwoodster_ / redrobot: ^^ the updated devstack CR will be ok to merge because it will work *alongside* the other method, so it won't break everything -- we can update the infra job once it's in19:02
rm_workand it makes testing changes in devstack MUCH simpleer19:03
jvrbanacrm_work, oooo hang on19:03
jvrbanacrm_work, derp
*** paul_glass has joined #openstack-barbican19:04
aleewoodster_, does changing the column size require a migration script?  or is that all handled with sqlalchemy?19:04
jvrbanacredrobot, woodster_, hockeynut, alee, rellerreller, could we get this merged?
rm_workI guess there is no way to do this without leaking information?19:06
aleewoodster_, redrobot ?  I ran the migration script but saw no changes19:06
rm_workbecause Ideally, you should get 403 whether a container exists or not, if you don't have access to that tenant's containers19:07
rm_workbecause getting a 404 vs a 403 leaks info (whether the container exists)19:07
rm_workbut if the container doesn't exist, there can't be ACLs for it19:07
rm_workso there's no way to tell if you'd normally have access or not >_>19:07
rm_workanyone else think that's a problem? and/or just one we'll have to live with? or am I super paranoid19:08
jvrbanacrm_work, I believe based on prior discussions 404 is the preferred way19:08
rm_workk, so we'll leak a minor amount of info, but i guess we don't care / can't work around that19:08
jvrbanacrm_work, soo ideally, we should return a 404 if they don't have permission as well19:09
rm_workah, heh19:09
rm_workyeah I guess actually that is the proper solution19:09
jvrbanacthat way you don't leak existence19:09
aleerellerreller, ^ any idea?19:10
jvrbanacalee, I believe changing the column size does require a migration19:10
aleejvrbanac, interesting -- I tried running the db script and saw nothing generated19:11
jvrbanacalee, The migration generation script is kind of easy to mess up. I mess it up all the time.19:11
jvrbanacUsually if it doesn't generate anything it's because the state of your db is newer19:12
rellerrelleralee I usually prefer not to mention anything at all. If item does not exist or if they do not have permissions then I usually return the same error, "you do not have permission for this object."19:12
jvrbanacrellerreller, I think that was for rm_work ;)19:13
aleejvrbanac, yeah - thats the steps I followed19:14
jvrbanactwo conversations going on at once lol19:14
rellerrellerIt's for everyone :) alee happened to ask me the question.19:14
aleerellerreller, you're answering rm_work question ..19:14
rellerrelleralee is the thing you wanted me to review, right?19:15
aleerellerreller, yes19:15
rellerrelleralee good. I have 15 free minutes now.19:15
redrobotrellerreller alee  so I'm not sure why we're trying to do DER->PEM conversions in Barbican.  Are there no libraries that can convert that for us?19:16
aleerellerreller, I discovered though that the gate was failing because it was trying to store the generated csr in barbican-meta table and failing because that table has  a limit of 25619:16
aleeso my question is -- changing from 256 -> sa.Text would presumably need a migration script , right?19:17
aleebut the db script returns no output.19:17
aleemaybe the script can't handle it ?19:18
aleeredrobot, we basically return the keys as DER - which is fine except that I need to use that key within barbican to generate a csr for the stored key case.19:19
aleeredrobot, the only way I can do that for the case where the stored key is passphrase encrypted is to convert to PEM and import that way19:20
jvrbanacalee, so you started with an empty db, started up Barbican in the old state, killed barbican, checked out your latest code, and then ran the script?19:20
redrobotalee my question was more about why we're manually doing the DER to PEM conversion, instead of using a library19:20
rm_workalee: yeah, we have some code in Octavia and Neutron for doing that kind of operation -- which is the stuff I wanted to live in Castellan but got shot down ;P19:20
aleejvrbanac, yup19:20
aleejvrbanac, let me try again ..19:21
rm_worklooks like you are now getting to experience the pain :P19:21
aleeredrobot, agreed it would be nicer to use a library -- follow on CR for liberty?19:22
jvrbanacrm_work, alee, redrobot, you might look at cryptography for that. They're starting to add x509 stuff in:
rm_workyeah that should be useful once it's complete19:24
aleejvrbanac, yes - thats the plan.  they dont have the ability to generate a csr yet.19:24
jvrbanacI'm sure they wouldn't mind PRs ;)19:24
redrobotalee for sure! ... I'm still drudging through every possible secret-type/content-type workflow, so I'll probably have some questions for rellerreller and you later.19:24
redrobotalee rellerreller if y'all get a chance, could you drop some comments on my first stab at the "symmetric" secret type?19:25
aleejvrbanac, the plan has always been to replace all this openSSL / conversion stuff with crytography when its ready19:25
redrobotalee rellerreller
rm_workredrobot: would you have a problem merging soon? it should pass checks now19:27
rm_workwant to get started on this as it's a three-parter19:27
redrobotrm_work part two is the infra CR?19:28
rellerrelleralee Do you have time to discuss your CR?19:28
rm_workredrobot: part two is Infra, part three is cleaning up the old method19:28
rellerrelleralee I'm curious why translations had to change.19:28
aleerellerreller, looking19:29
redrobotrm_work sounds like a fine approach.  be sure to add "Depends-on: I0ec63819b3aae21a6ffaed5cf8285e26dce6ae94" to the infra CR19:29
aleerellerreller, which part?19:30
rm_workredrobot: that's only if I submit the infra CR before this merges, right?19:30
aleerellerreller, first off --- all the translations code was run without any '\n''s in it in the pem functions19:30
redrobotrm_work yeah... are you going to wait for this CR to submit to infra?  I would submit now since they usually take a couple of days to merge things19:31
aleerellerreller, that resulted in some faulty logic once the '\n's were added19:31
rellerrelleralee get_secret now takes parameter for pem_needed, but I don't see why that is needed.19:31
rm_workredrobot: hmm k19:31
aleerellerreller, ok - so the idea here is that in the stored key case I need to take the key and generate a csr.19:32
aleerellerreller, now the way I do that is openssl.crypto.load_privatekey()19:33
rellerrelleralee Stored key case is when generate a CSR that is signed with key that already exists in Barbican???19:33
aleerellerreller, so I need to load this key in PEM format.19:34
rellerrelleralee And default return from get_secret is binary, right?19:35
rellerrellerSorry, I was on vacation and tried to forget about all of this for a while ;)19:35
aleerellerreller, I need a vacation now too ..19:35
rellerrellerSo you want to have extra parameter to return in PEM format and then use it load and sign CSR. Got it.19:36
aleeyup -- in anything, this is a precursor to the ability to request the rerturn format of the secret19:36
aleerather than just returning binary19:36
rellerrelleralee I like that19:37
redrobotalee rellerreller  so, the return format of the secret is listed in the content_types property19:37
alee(except that we'll consider exposing that to the api in liberty)19:37
rellerrelleralee is there any code in your CR that invokes this new functionality? I cannot seem to find any. Will there be a subsequent patch?19:37
aleeredrobot, the way things are implemented - we always return binary19:38
rellerrellerredrobot That is not honored. You can store a key in base64 content type, but it will always be returned in binary.19:38
aleerellerreller, see certificate_resources.py19:38
redrobotalee rellerreller  base64 is an encoding, not a content type.19:38
openstackgerritMerged openstack/barbican: Return container not found before ACL checks
aleeredrobot, this is why we need you to do your investigation :)19:39
redrobotalee rellerreller base64 as an encoding was introduced to allow non-ascii characters inside the JSON request.19:39
rm_workredrobot: err, do you happen to know WHERE the code is in infra for the dsvm job?19:39
rm_workredrobot: I was assuming it'd be here:
rm_workbut it looks like not19:40
redrobotrm_work that is the correct file for the barbican devstack jobs.  We only have two.  The first one is the one that always runs, the -f21 one is the one that does dogtag and is still experimental19:41
redrobotrm_work what were you expecting to find?19:41
rm_workredrobot: i don't see how it's deciding where to pull barbican from19:41
rm_workit should be cloning barbican and moving files around and stuff19:41
redrobotrm_work these two lines tell devstack to clone barbican
redrobotrm_work and also clone python-barbicanclient19:42
rm_workyeah but something also tells it to copy the files around from ./contrib/devstack/19:42
redrobotrm_work yes, after it's cloned by devstack, the pre_test_hook moves the lib and extras.d files to the devstack structure here:
rm_workOH, it's part of your own code19:43
rm_worklooool ok19:43
rm_workmaybe I need to do more research, might be able to do it in one step19:44
redrobotrm_work cool cool, glad I could help :)19:44
rm_workor actually, no, but that will need to change19:44
redrobotrm_work does that secure your vote for me for PTL?  :D19:44
rm_workheh, are you running opposed this time? :P19:44
redrobotrm_work nope, but I still feel like I need to campaign19:44
rm_workwoo and devstack is broken19:48
openstackLaunchpad bug 1441820 in devstack " fails because /opt/stack/requirements/global-requirements.txt isn't present" [Undecided,In progress] - Assigned to Amrith (amrith)19:48
rm_work^^ merging soon19:48
woodster_alee, I see you comments about migrations above....19:49
woodster_alee, I would think you need to alter the column type in the migration script.19:50
rellerrelleralee I reviewed the CR. I guess I only have the dead code comment. I would like to see the is_public_key_valid implemented, but I do not see a way load the public key using the OpenSSL library you use for is_private_key_valid.19:51
woodster_any Python logging experts out there? Stevedore isn't loading a plugin because of an exception the plugin __init__ throws, but the stack trace isn't showing up in logs even though they do a log.exception. Do I really need to do a logging.getLogger('strevedpre)19:56
woodster_...sort of thing?19:56
redrobotwoodster_ I would first make sure that the logging level of your process matches the logging level of the stevedore thing.  maybe your logging level is higher than what stevedore is spitting out?19:57
woodster_ logging.getLogger('strevedore').setLevel(logging.INFO) that is19:57
aleerellerreller, right - thats why I left it for now.19:57
aleerellerreller, we can revisit when cryptography is ready19:58
woodster_My running app is at DEBUG level, but import foo on the plugin manager module might be grabbing the logger at a higher level?19:58
aleerellerreller, the private key is really whats important in any case --- the public key can be derived from it19:58
aleerellerreller, I'll remove the dead code19:58
aleewoodster_, I'm trying this --- but its not working so far ..19:59
woodster_redrobot, rellerreller, kfarr that is essentially the issue with Christopher in the maining list, trying to use the KMIP plugin19:59
woodster_alee, you might want to examine the actual schema with psql or some such tool. try to upgrade/downgrade versions to see how the schema changes. The autogen tool is lmited, missing things like default values for new non-nullable columns for example/20:00
woodster_alee, that looks good. Just try to upgrade/downgrade to/from that version file and verify the schema changes a expected, ideally with some data already in the database when you do it20:02
woodster_alee, the clob > 255 to string(255) might crash though?20:02
*** nickrmc83 has quit IRC20:02
kfarrwoodster_ glad you figured that out!  I was stumped20:04
*** nickrmc84 has joined #openstack-barbican20:04
kfarrwoodster_ Which section of the code is the part that is failing but not showing up in the logs?20:05
woodster_kfarr, yeah it is super annoying! My local setup is borking because KMIP is looking for a cert file I don't have, but stevedore traps/logs the exception and then doesn't load the module. So barbican just thinks there are no modules loaded20:05
woodster_kfarr, this line:
*** paul_glass1 has joined #openstack-barbican20:07
*** nkinder has joined #openstack-barbican20:09
aleewoodster_, well I'm trying the upgrade using the script -- but its failing -- I''m using this ..20:09
alee./bin/ --dburl sqlite:////var/lib/barbican/barbican.sqlite upgrade -v head20:09
aleeand it fails with : OperationalError: (OperationalError) near "ALTER": syntax error u'ALTER TABLE order_barbican_metadata ALTER COLUMN value TYPE TEXT' ()20:10
*** paul_glass has quit IRC20:10
kfarrwoodster_ Where is the log.exception?20:12
*** dave-mccowan has joined #openstack-barbican20:14
woodster_alee, so are you going from the previous db version to the latest one then?20:14
aleewoodster_, yup20:14
woodster_kfarr, this log.exception is from the stevedore side20:14
rellerrelleralee I will be good with CR once dead code is removed. Looks good :)20:14
aleewoodster_, just a single script20:14
aleerellerreller, cool -- trying to add the migration script/db change in there20:15
alee(have already removed dead code)20:15
woodster_alee, so maybe type alterations require difference syntax? too bad it doesn't auto generate that for you :\20:15
aleeyeah -- looks right though --
woodster_alee, maybe try a column name alter first to see if that works correctly20:20
aleewoodster_, ok - maybe it just doesn;t like me .. OperationalError: (OperationalError) near "value": syntax error u'ALTER TABLE order_barbican_metadata RENAME value TO foo' ()20:22
aleewoodster_, can you try it on your db?20:23
redrobotalee is it possible that it's just not supported in sqlite?   I recall someone mentioning that migrations in sqlite are not possible.20:23
aleethat would be weird -- I fell like I've done migrations efore ..20:24
redrobotIIRC (and there's a good chance that I don't)  You can add new tables to sqlite databases, but editing existing tables is a no-go20:25
redrobotalee "SQLite supports a limited subset of ALTER TABLE"
aleeredrobot, ok - thats likely it then20:28
*** crc32 has joined #openstack-barbican20:30
woodster_alee, sorry I thought you been doing this with a postgres db for some reason. The migration script used to warn you if you were using sqlite though :\20:31
alee WARNING barbican.model.migration.commands [-] !!! Limited support for migration commands using sqlite databases; This operation may not succeed.20:31
aleewoodster_, yeah - who heeds warnings?20:31
aleethats like reading the manual ..20:32
woodster_alee, ha!20:33
aleewoodster_, rellerreller , redrobot - new version being uploaded momentarily20:41
openstackgerritAde Lee proposed openstack/barbican: Changes to get remaining cert functional tests working
aleeit would be super sweet if we could get this in today20:42
*** rellerreller has quit IRC20:45
*** dfinnrml has joined #openstack-barbican20:53
*** dimtruck is now known as zz_dimtruck20:54
jvrbanacredrobot, soo it looks like our devstack gate is functioning now; however, there seems to be a weird problem with the neutron gate that has nothing to do with us.20:55
redrobotjvrbanac ugh...20:56
*** zz_dimtruck is now known as dimtruck20:57
openstackgerritAdam Harwell proposed openstack/barbican: Use the new Devstack external plugin method
rm_worklet's see if this works :P20:58
iguethsCan someone poke at some point? Latest patchset is considerably smaller than what originally went in.20:58
kfarrHey all, we were encountering what might be a bug when trying to create and retrieve an asymmetric key pair:
openstackLaunchpad bug 1441848 in python-barbicanclient "unexpected keyword argument 'sub_status' when getting an order" [Undecided,New]20:58
kfarrAm I missing something obvious?20:59
rm_workjvrbanac: yeah devstack in gate is f'd21:00
redrobotkfarr asymmetric key pair in Barbican proper?  I don't think the client has support for typed secrets yet21:00
jvrbanackfarr, however, that error you're seeing about sub_status message should be resolved in the CR I linked21:01
jvrbanacrm_work, :(21:02
kfarrredrobot, we were using python-barbicanclient21:02
kfarrjvrbanac, I'll see if your patch fixes it, (it looks like it'll work) thanks!21:02
rm_workjvrbanac: waiting for to merge21:03
rm_worksorry wrong one21:03
kfarrjvrbanac, redrobot, fixes it, thanks!21:07
jvrbanackfarr, np21:07
redrobotanyone have a working barbican right now?21:16
redrobotcan someone run these commands and let me know if it works?21:16
jvrbanacredrobot, 400 bad request21:17
redrobotjvrbanac crap.  I think that's a bug21:18
aleeredrobot, woodster_ rellerreller -- looks like it passed the gate this time21:24
aleeredrobot, woodster_ rellerreller, jaosorior -- waiting for some +2's ..21:25
aleejvrbanac, hockeynut ^^21:25
redrobotalee will look at it.  also, does this sequence of steps look correct to you for a one POST request:
aleeredrobot, interesting -- so you base64 encode the entire thing including headers and footers21:28
redrobotalee yes, that's what we do with asymmetric keys.   An alternative would be to json escape the newlines and call the content type "text/plain" ?21:29
aleeredrobot, I think you need to put together a set of scripts like this that you think would correspond to valid inputs - and then we can discuss.21:31
aleeredrobot, thats what the google hangout was supposed to be about21:31
redrobotalee yep, but going through every scenario is taking way longer than I expected21:32
openstackgerritAdam Harwell proposed openstack/barbican: Use the new Devstack external plugin method
openstackgerritwerner mendizabal proposed openstack/barbican: Create Barbican python scripts for development.
aleeredrobot, well - the plus side is that these scripts form the basis of documentation21:32
aleeredrobot, put together what you think should be valid -- and then we can decide if it actually is.21:33
redrobotalee I'm actually writing the docs as I go... killing two birds with one stone if you will.21:33
aleeredrobot, and we can attempt to do tings like create a csr from it for the stored key case21:33
aleeredrobot, thats probably the best test case -- 1) store public, private key, passphrase  2) use them to generate a CSR in the stored key case21:36
aleethat way we can see if we can actually use what is stored internally21:38
redrobotalee agreed21:38
aleeredrobot, of course to test all this out - we need my patch in first :)21:38
aleehint, hint, nudge, nudge ..21:39
*** chlong has joined #openstack-barbican22:01
*** dimtruck is now known as zz_dimtruck22:03
*** jamielennox is now known as jamielennox|away22:11
*** paul_glass1 has quit IRC22:11
openstackgerritJohn Vrbanac proposed openstack/python-barbicanclient: Raising errors from the client instead of keystone
openstackgerritAdam Harwell proposed openstack/barbican: Use the new Devstack external plugin method
dave-mccowanalee, woodster_ any thoughts on this refactoring CR?  I have one more patch for validators, and want to know which code to base it on.22:41
*** SheenaG has left #openstack-barbican22:50
openstackgerritDave McCowan proposed openstack/barbican: Add Functional Tests for Private Key Secret Type
*** kebray has quit IRC22:56
openstackgerritDouglas Mendizábal proposed openstack/barbican: Document Symmetric Secret Type
kfarrwoodster_ ping22:56
openstackgerritCharles Neill proposed openstack/barbican: Security tests for Order resources
openstackgerritDouglas Mendizábal proposed openstack/barbican: Document public secret type
*** chlong has quit IRC23:05
openstackgerritJohn Wood proposed openstack/barbican: Expose root cause plugin exceptions
woodster_kfarr, hello23:11
*** dfinnrml has quit IRC23:12
*** jamielennox|away is now known as jamielennox23:12
kfarrwoodster_ I've been thinking about what could be done to fix that logging issue you mentioned earlier.  Have you come to any conclusions?23:12
woodster_kfarr, ha, I just put up a CR for that:
woodster_kfarr see what you think though, to see if there's a better way to deal with that23:13
kfarroh, cool!23:14
woodster_kfarr, for some reason the logger on stevedore is being set to disabled even if I get its logger right before I call the plugin manager enable it. So the exception logging it does fails. The bigger problem though is that it keeps on processing and returns to barbican. It isn't until barbican tries to locate a plugin later that you see an error.23:15
woodster_kfarr, so config issues with plugins should fail fast so you know as early as possible soething is misconfigured23:15
openstackgerritAdam Harwell proposed openstack/barbican: Use the new Devstack external plugin method
kfarrwoodster_ Great!  That seems like a good solution.  I didn't know if it was fixable and was wondering if we just needed to move an error-creating code out of the plugins. I like your code better23:17
woodster_kfarr, I was hoping to keep the plugins as simple as possible, and hard-broken config issues a plugin detects should be raised as an exception imho23:18
woodster_rm_work, are you trying to get a clean CR for the devstack stuff?23:19
openstackgerritJohn Vrbanac proposed openstack/python-barbicanclient: Raising errors from the client instead of ksclient
*** kebray has joined #openstack-barbican23:33
*** jkf has quit IRC23:44
openstackgerritMerged openstack/barbican: Imported Translations from Transifex
openstackgerritJohn Wood proposed openstack/barbican: Add retry server and functional tests to DevStack

Generated by 2.14.0 by Marius Gedminas - find it at!