Tuesday, 2015-04-14

*** kebray has joined #openstack-barbican00:05
*** dave-mccowan has joined #openstack-barbican00:23
*** dave-mccowan has quit IRC00:29
*** SheenaG has quit IRC00:31
*** SheenaG has joined #openstack-barbican00:32
*** gyee has quit IRC00:35
*** SheenaG has quit IRC00:41
*** SheenaG has joined #openstack-barbican00:41
*** dimtruck is now known as zz_dimtruck00:46
*** SheenaG has quit IRC01:11
*** zz_dimtruck is now known as dimtruck01:25
*** kebray has quit IRC01:36
*** SheenaG has joined #openstack-barbican02:08
elmikoanyone around?02:10
*** igueths has quit IRC02:10
*** rm_work|away is now known as rm_work02:52
*** rm_work is now known as rm_work|away03:05
*** dave-mccowan has joined #openstack-barbican03:14
*** xaeth_afk is now known as xaeth03:19
openstackgerritDave McCowan proposed openstack/barbican: Fix expectations of order certificate test cases  https://review.openstack.org/17316303:42
*** xaeth is now known as xaeth_afk03:45
*** SheenaG has left #openstack-barbican03:51
*** rm_work|away is now known as rm_work03:52
*** xaeth_afk is now known as xaeth04:03
*** woodster_ has quit IRC04:10
*** rm_you| has joined #openstack-barbican04:32
*** rm_you has quit IRC04:35
*** alee_afk has quit IRC04:36
*** alee_afk has joined #openstack-barbican04:36
*** woodster_ has joined #openstack-barbican04:53
*** xaeth is now known as xaeth_afk05:01
openstackgerritChelsea Winfree proposed openstack/python-barbicanclient: Adding raw and clif formatted payload  https://review.openstack.org/17317205:12
openstackgerritOpenStack Proposal Bot proposed openstack/barbican: Imported Translations from Transifex  https://review.openstack.org/17318006:07
*** dimtruck is now known as zz_dimtruck06:23
*** dave-mccowan has quit IRC06:58
*** jamielennox is now known as jamielennox|away07:02
*** woodster_ has quit IRC07:20
*** chlong has quit IRC07:27
*** tkelsey has joined #openstack-barbican08:19
*** zz_dimtruck is now known as dimtruck08:52
*** jaosorior has joined #openstack-barbican08:53
*** dimtruck is now known as zz_dimtruck09:02
openstackgerritJuan Antonio Osorio Robles proposed openstack/barbican: Enable alternate error message for OpenSSL 1.0.2  https://review.openstack.org/17284410:46
*** zz_dimtruck is now known as dimtruck11:12
openstackgerritJuan Antonio Osorio Robles proposed openstack/barbican: Readability-related changes to secret store functions  https://review.openstack.org/17237811:27
*** dimtruck is now known as zz_dimtruck11:35
*** woodster_ has joined #openstack-barbican12:15
*** rellerreller has joined #openstack-barbican12:34
*** rellerreller has quit IRC13:04
*** zz_dimtruck is now known as dimtruck13:10
elmikoredrobot: hey, i ran through the migrations with a true mysql server last night. it did not go so well13:19
*** nkinder has joined #openstack-barbican13:39
*** alee has joined #openstack-barbican13:45
*** joesavak has joined #openstack-barbican13:50
*** stanzi has joined #openstack-barbican13:50
*** dimtruck is now known as zz_dimtruck14:01
*** zz_dimtruck is now known as dimtruck14:13
*** paul_glass has joined #openstack-barbican14:15
*** xaeth_afk is now known as xaeth14:23
openstackgerritMerged openstack/barbican: Imported Translations from Transifex  https://review.openstack.org/17318014:34
*** stanzi has quit IRC14:42
*** stanzi has joined #openstack-barbican14:42
*** stanzi_ has joined #openstack-barbican14:46
*** stanzi has quit IRC14:47
*** dave-mccowan has joined #openstack-barbican14:51
*** xaeth is now known as xaeth_afk14:51
*** nkinder has quit IRC14:58
*** xaeth_afk is now known as xaeth14:59
*** rellerreller has joined #openstack-barbican15:04
*** joesavak has quit IRC15:04
*** darrenmoffat has quit IRC15:12
*** darrenmoffat has joined #openstack-barbican15:13
*** stanzi_ has quit IRC15:16
*** stanzi has joined #openstack-barbican15:17
elmikoredrobot: you around?15:17
*** xaeth is now known as xaeth_afk15:19
*** xaeth_afk is now known as xaeth15:19
*** stanzi has quit IRC15:20
*** stanzi has joined #openstack-barbican15:20
redrobotelmiko o/15:26
elmikoredrobot: hey, ran into a bunch of weird issues with mysql15:26
elmikoseems that true mysql is a little more strict than maria15:26
elmikoi wanted to confirm something though,15:27
elmikoin stable/juno, the db schema gets create when a call comes in? is there another way to create the schema?15:27
redrobotelmiko so, I think there's a config option that controls whether the API attempts to build the schema or not15:30
elmikooh right, the auto create field15:31
*** SheenaG has joined #openstack-barbican15:31
elmikoredrobot: so, the big issues i am running into with mysql are that drop_constraints and alter_column seem to want type information15:31
elmikothe alembic docs mention this for the drop_constraints method15:31
elmikoi've fixed a couple of migrations on a local branch, but now i'm running into a new sticking point15:32
elmikoredrobot: i think it's probably worthwhile to have someone else take a look at this too. i'm concerned that some of my fixes are a little too hacky.15:34
elmikoi'm at the edge of my sql knowledge, and there are some odd constraint keys that are getting creating automagically that i'm not sure about15:35
redrobotelmiko sure... let me ping woodster_ and see if he has time to poke at that stuff15:36
redrobotelmiko he's way better at db stuff than I am15:36
elmikoredrobot: ok, cool. i can pass on what i've found to him. maybe it's a little more simple than i imagine15:36
dave-mccowanredrobot, ping.15:38
redrobotdave-mccowan pong15:39
dave-mccowanredrobot, how's it going with fixing the b64 encoding?  let me know if you can help.  i have a fix for #1443008, but the test case needs all the b64 stuff straight to pass.15:40
redrobotdave-mccowan almost done... I had to touch a bunch of stuff and fix a lot of functional tests.15:41
redrobotdave-mccowan I just have the stored-key case left.15:41
*** gyee has joined #openstack-barbican15:42
dave-mccowanredrobot, awesome!  i've become very familiar with that code.  just let me know if you'd like any early review or extra testing.15:42
redrobotdave-mccowan I can push a WIP right now if you want to take a look at it15:43
dave-mccowanredrobot, sure.15:43
aleeredrobot, dave-mccowan - let me know when ya'll think you have something ready to work together, and I'll test it against dogtag15:44
woodster_elmiko: redrobot mentioned you were dealing with mysql migrations?15:44
elmikowoodster_: yea, running into a bunch of weird issue15:46
elmikowoodster_: https://github.com/openstack/barbican/blob/master/barbican/model/migration/alembic_migrations/versions/795737bb3c3_change_tenants_to_projects.py15:46
elmikothat migration is where my problems started15:46
*** stanzi has quit IRC15:46
woodster_elmiko: I'm wondering if the 'InnoDB' lines in the models.py module are breaking things for mysql....15:46
openstackgerritDouglas Mendizábal proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST  https://review.openstack.org/17339615:46
redrobotdave-mccowan alee  ^^  it's going to fail the stored-key case15:47
*** stanzi has joined #openstack-barbican15:47
elmikowoodster_: hmm, not sure15:47
*** stanzi_ has joined #openstack-barbican15:48
rellerrellerredrobot I am also interested when you are done. I'm glad others are seeing how much fine content types are!15:48
elmikowoodster_: what other options could i try?15:49
elmikowoodster_: seems like the main complaint i'm getting to start with is that drop_constraints and alter_column want to know the types in mysql15:49
woodster_elmiko, well, there are sqlalchemy calls that can be made for migrations as well (instead of alembic op)...maybe try to compare to some other version files15:49
redrobotrellerreller haha yeah, some of that stuff is like a rube goldberg machine15:50
elmikowoodster_: fixing the type information definitely helped15:50
redrobotrellerreller I want to refactor it, but I'm trying to keep this CR small, so I'll probably come back around after everything is working and simplify a lot of it.15:50
elmikowoodster_: but i'm getting a little turned around about the constraints because it seems there are hidden constraint columns that are being created and i'm having to remove them by name15:51
*** dave-mccowan has quit IRC15:51
*** stanzi has quit IRC15:51
*** stanzi_ has quit IRC15:51
*** stanzi has joined #openstack-barbican15:51
woodster_elmiko: check out 1a7cf79559e3_new_secret_and_container_acl_tables.py15:54
woodster_elmiko: it uses the sqlalchemy foo15:56
elmikowoodster_: yea, i looked at that one. i'll study further15:56
*** dave-mccowan has joined #openstack-barbican15:57
woodster_elmiko: please let me know what you discover. I'm in various sprint planning meetings today but will be able to help out later this afternoon16:01
woodster_elmiko: I'll be on IRC throughout though if you discover something16:01
rellerrellerredrobot I put some comments on https://review.openstack.org/17339616:02
*** kebray has joined #openstack-barbican16:02
elmikowoodster_: ack, i'll try to collect my results16:02
redrobotrellerreller thanks!16:03
*** joesavak has joined #openstack-barbican16:04
*** kebray has quit IRC16:06
*** joesavak has quit IRC16:11
*** stanzi has quit IRC16:20
*** stanzi has joined #openstack-barbican16:21
*** stanzi has quit IRC16:26
*** xaeth is now known as xaeth_afk16:31
*** xaeth_afk is now known as xaeth16:38
*** joesavak has joined #openstack-barbican16:43
*** stanzi has joined #openstack-barbican16:43
dave-mccowanredrobot, w00t.   Ran 18 tests in 3.775s    OK16:46
*** rellerreller has quit IRC16:51
*** dimtruck is now known as zz_dimtruck16:51
*** stanzi has quit IRC16:53
*** stanzi has joined #openstack-barbican16:53
*** stanzi_ has joined #openstack-barbican16:54
*** stanzi has quit IRC16:54
elmikowoodster_: so yea, looks like the issues i'm having with mysql are involving the extra columns that are created with respect to the constraints.16:56
*** xaeth is now known as xaeth_afk16:58
*** zz_dimtruck is now known as dimtruck16:58
*** joesavak has quit IRC17:05
*** stanzi_ has quit IRC17:06
*** stanzi has joined #openstack-barbican17:07
*** kebray has joined #openstack-barbican17:08
*** stanzi has quit IRC17:11
*** joesavak has joined #openstack-barbican17:14
*** SheenaG has quit IRC17:14
*** alee is now known as alee_lunch17:17
*** rellerreller has joined #openstack-barbican17:31
*** tkelsey has quit IRC17:33
*** SheenaG has joined #openstack-barbican17:37
*** tkelsey has joined #openstack-barbican17:37
*** stanzi has joined #openstack-barbican17:37
*** tkelsey has quit IRC17:43
*** stanzi has quit IRC17:46
*** jaosorior has quit IRC17:52
redrobotdave-mccowan hey I think one of the rsa smoke tests is incorrect17:58
redrobotdave-mccowan https://github.com/openstack/barbican/blob/master/functionaltests/api/v1/smoke/test_rsa.py#L83517:58
redrobotdave-mccowan the CSR has newlines in it, so it can't be just included in the request17:58
dave-mccowanredrobot, yes.  i've got a patch that fixes those issues.17:58
*** stanzi has joined #openstack-barbican17:58
dave-mccowanredrobot, waiting for +2s17:58
*** stanzi has quit IRC17:59
redrobotdave-mccowan link?17:59
redrobotdave-mccowan are we going to base64(CMC) then?  The functional test right now is doing json.dumps(CMC)17:59
*** stanzi has joined #openstack-barbican17:59
dave-mccowanredrobot https://review.openstack.org/#/c/173163/18:00
dave-mccowanredrobot, also https://review.openstack.org/#/c/172819/18:01
dave-mccowanredrobot,   i have a delta from your patch that now has all 18 tests working.  i need to figure out how to upload the good parts.18:01
dave-mccowanredrobot, yes it should be base64(cmc).18:03
dave-mccowanredrobot, what git commands can i use to upload my patchset to your CR?18:05
*** jamielennox|away is now known as jamielennox18:05
redrobotdave-mccowan yours depends on mine?18:06
dave-mccowani got the stored-key case working on a branch that i fetched from your CR.18:06
dave-mccowantest_rsa.py works (but it breaks 20 unit tests, and 10 functional tests, so more work to do)18:07
redrobotoh I see... well hmmm.... so the problem is that mine won't pass the gate18:07
redrobotso we'd have to squash it into a single commit.18:07
redrobotwant to add the changes to my CR?18:08
*** alee_lunch is now known as alee18:08
redrobotjust git commit --amend, and it'll show both of us as authors.18:08
dave-mccowani must have check it out wrong... it's trying to amend to another CR18:09
*** nkinder has joined #openstack-barbican18:13
redrobotdave-mccowan git review -d 17339618:15
openstackgerritDave McCowan proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST  https://review.openstack.org/17339618:19
dave-mccowanredrobot, if you agree with those changes, we can split fixing unit and functional tests.18:20
redrobotdave-mccowan looking18:25
*** xaeth_afk is now known as xaeth18:26
openstackgerritDave McCowan proposed openstack/barbican: Fix functional test test_rsa_order_certificate_from_csr  https://review.openstack.org/17345218:33
woodster_elmiko: so the MySQL column issue happened even with sqlalchemy correct?18:36
*** stanzi has quit IRC18:37
*** stanzi has joined #openstack-barbican18:37
openstackgerritDave McCowan proposed openstack/barbican: Fix generating a CSR with an encrypted private key  https://review.openstack.org/17345718:39
*** stanzi has quit IRC18:41
*** stanzi has joined #openstack-barbican18:42
woodster_Can cores review these two small CRs?: https://review.openstack.org/171868  and  https://review.openstack.org/16994618:46
elmikowoodster_: well, i'm sorting through this18:46
elmikowoodster_: it looks like there are some constraints that get automatically named depending on the db impl18:47
elmikowoodster_: these needs to be deleted for mysql, but first their names need to be determined18:47
elmikoi'm still understanding what these extra constraints are though18:48
*** stanzi has quit IRC18:48
elmikowoodster_: also, this migration is problematic https://github.com/openstack/barbican/blob/master/barbican/model/migration/alembic_migrations/versions/1c0f328bfce0_fixing_composite_primary_keys_and_.py#L3918:49
*** stanzi has joined #openstack-barbican18:49
elmikothat highlighted line is using a column name that is postgresql specific as far as i can tell18:49
elmikowoodster_: like, that column preferred_certificate_authorities_project_id_key is the name generated for postgres by sqlalchemy, but as near as i can tell it would be preferred_certificate_authorities_project_ibfk_1 for mysql18:51
*** stanzi has quit IRC18:51
elmikoso... basically, fun times ;)18:51
*** stanzi has joined #openstack-barbican18:51
redrobotdave-mccowand looks good.  I'm going to remove the skip for one of the tests18:54
redrobotdave-mccowan stored key is still failing for me though18:55
dave-mccowanredrobot, i think i missed i file when moving between branches.18:55
woodster_elmiko: so maybe part of the issue is using alembic's op instead of sqlalchemy. Maybe we could then genetically name our constraints and sqlalchemy maps to database specific names?18:57
elmikowoodster_: i think that would help, but i'm still not quite understanding why these extra constraints only cause an issue with mysql18:58
*** paul_glass has quit IRC18:58
openstackgerritDave McCowan proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST  https://review.openstack.org/17339619:00
dave-mccowanredrobot, i missed my changes to translations.py  ^^19:00
*** paul_glass has joined #openstack-barbican19:01
hockeynuttdink jvrbanac would y'all be so kind as to take a look at https://review.openstack.org/#/c/172604/ and hit me with your comments?19:02
elmikowoodster_: the annoying thing is that for postgresql the op.drop_constraints call seems to get rid of the associated constraints fields. but for mysql they need to be removed by name.19:10
*** dave-mccowan has quit IRC19:10
elmikowoodster_: take a gander at https://gist.github.com/elmiko/9badea7384aaec355a0619:14
elmikoon mysql, the tenant_secret_ibfk_1 foreign key constraint needs to be deleted before the _tenant_secret_uc unique can be dropped19:14
elmikoon postrgresl this doesn't seem to matter19:15
elmikolikewise, in the 1c0f328bfce0 migration, the drop_constraint needs extra work for mysql19:16
woodster_elmiko: my hope is that we can use sqlalchemy only, and give the constraint a name like 'tenant_secret_uc' and then sqlalchemy translates to the db specific naming under the hood19:17
woodster_elmiko: so we would need to name all constraints though19:18
elmikowoodster_: it seems the actual unique constraints do have names assigned in sqlalchemy, alembic seems to complain about the implied, and unnamed, foreign key constraints19:18
elmikowoodster_: yea, i think so19:18
elmikowoodster_: some of them are named though19:19
woodster_elmiko: these names would need to be synced with those in models.py too19:20
elmikowoodster_: that makes sense, this all just smells really fishy because the error i get when removing the named constraint (_tenant_secret_uc) is a complaint about it being used in another constraint (tenant_secret_ibfk_1)19:22
woodster_elmiko: yeah that is odd19:23
woodster_elmiko: my hunch is we need consistent and unique constraints everywhere19:24
woodster_elmiko: ....or else things get out of whack19:24
redrobotdave's not here man19:30
elmikowoodster_: i *think* if you look at the models.ProjectSecret that the project_id field which contains a ForeignKey constraint in the schema, that it would need a name19:33
*** dave-mccowan has joined #openstack-barbican19:33
elmikowoodster_: or, it would be need to be added as a constraint separately19:33
elmikowoodster_: then, they could be dropped consistently19:33
woodster_elmiko: I think that's the case. My guess is other openstack projects are doing this too19:35
elmikowoodster_: i'm looking at the sahara base to see what it does, but there aren't any migrations with explicitly named constraints being dropped19:36
elmikowoodster_: i think the rename is just tricky19:36
elmikowoodster_: i'll keep playing around with this though19:36
*** dave-mccowan has quit IRC19:37
*** SheenaG has quit IRC19:39
*** rellerreller has quit IRC19:40
redrobotdoes anyone have time to review https://review.openstack.org/#/c/172714/3 ?19:45
redrobotI need it to rebase my work on top of the openssl 1.0.2 fix19:45
redrobotwoodster_ you got time to workflow? https://review.openstack.org/#/c/172714/319:46
redrobotwoodster_ should be a quick review19:47
*** SheenaG has joined #openstack-barbican20:01
openstackgerritDouglas Mendizábal proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST  https://review.openstack.org/17339620:03
*** dimtruck is now known as zz_dimtruck20:04
*** stanzi has quit IRC20:07
*** stanzi has joined #openstack-barbican20:08
*** zz_dimtruck is now known as dimtruck20:09
*** stanzi has quit IRC20:12
*** igueths has joined #openstack-barbican20:14
*** dave-mccowan has joined #openstack-barbican20:14
openstackgerritMerged openstack/barbican: Sign CSRs issued in SnakeOilCA tests  https://review.openstack.org/17271420:16
openstackgerritMerged openstack/barbican: Enable alternate error message for OpenSSL 1.0.2  https://review.openstack.org/17284420:17
iguethsHey all, could I get some reviews on https://review.openstack.org/170693?20:19
redrobotanyone here familiar with pykmip?20:20
*** tkelsey has joined #openstack-barbican20:20
*** crc32 has joined #openstack-barbican20:22
*** tkelsey has quit IRC20:24
redrobotalee ping?20:29
redrobotkfarr around?20:29
elmikowoodster_: think i'm making some progress20:34
woodster_elmiko: hey, I'm back from meetings now...good timing then? ;)20:35
elmikohehe yea20:35
woodster_elmiko: is naming all the constraints the answer?20:35
elmikowoodster_: looks like it20:35
elmikowoodster_: it's working for both postgresql and mysql, now i'm worried that the fk constraints aren't getting replaced after the rename20:36
woodster_elmiko: so hopefully no need to replace the alembic op calls with sqlalchemy ones then?20:36
elmikowoodster_: hopefully /me fingers-crossed20:36
elmikowoodster_: if i get this patch working maybe i can share it and we can figure out how best to put it up for review?20:36
woodster_elmiko: that sounds good. So I imagine a few of those version files will need to be tweaked, maybe the models.py as well?20:37
elmikowoodster_: yup, both20:38
*** igueths has quit IRC20:42
woodster_elmiko: wow, so yeah maybe better to separate those into separate smaller CRs then20:43
elmikowoodster_: it's gonna be complicated...20:43
*** igueths has joined #openstack-barbican20:44
woodster_elmiko: can you just do it one entity/model at a time...so each one just making part of the overall migration work correctly20:44
elmikowoodster_: logically, i think we could do a review for the model changes + associated migrations, then the additional migration fixes20:44
elmikowoodster_: i think so20:44
redrobotelmiko woodster_  will this involve editing existing migrations?20:45
elmikoredrobot: unfortunately yea20:45
redrobotelmiko yikes...  well, that'll be fun to sort out for our deployment.20:45
elmikoredrobot: it also means that existing stable/juno -> kilo migration is probably not realistic20:45
woodster_redrobot, yeah so it turns out postgres is more forgiving than mysql20:45
redrobotwoodster_ think we can get away with rebuilding PROD?20:45
aleeredrobot, >20:46
aleewhats up?20:46
woodster_redrobot, do you mean our internal prod?20:46
elmikoredrobot: we can make it work but it will mean more introspection of db columns to make the migration work well20:46
redrobotalee was wondering if you knew something about KMIP, but I think I guessed correctly20:46
aleedave-mccowan, redrobot - hows the bug fixes goign?20:46
redrobotwoodster_ yeah...  if we edit existing migrations, then our current deployment won't match what would result from the new migrations20:47
aleedave-mccowan, redrobot you guys have something reaady for me to test?20:47
redrobotalee almost, https://review.openstack.org/#/c/173396/ is passing all the functional tests, but I still need to fix one more unit test20:47
elmikoredrobot, woodster_, i think there are 2 issues here. 1. existing deployments that need migration, 2. fixing the migrations/models for future stuff20:48
aleeredrobot, ok - this is a merge of both yours and dave-mccowan changes?20:48
redrobotalee yep20:48
woodster_redrobot, elmiko So it seems to me that the changes just tighten up the naming of constraints and so forth which shouldn't affect current schema unless a new name is added that wasn't there before?20:48
aleeredrobot, ok cool - I'll test and review it late tonight20:48
dave-mccowanalee, redrobot to get all the tests to pass, i have 4 more CRs outstanding too.  we'll need all 5 together.20:49
elmikowoodster_: that sounds about right20:49
alee4 more ..20:49
elmikowoodster_: also, the anonymous foreignkey constraints need to be named to prevent the bifurcation between mysql postgresql impls in the future20:49
elmikowoodster_, redrobot, so... it might be necessary to do this the hard way if we need to run the migrations on juno deployments. sounds like that might be a goal here?20:50
dave-mccowanalee, redrobot.  the 5th patch is a collaboration.  the guts is all there, and needs review, but there's about 30 test cases that need to be updated with expections of our clarified b64 design.  i've started fixing common/ and plugin/ unit test failures.20:50
redrobotelmiko yes, the main goal is to fix Juno -> Kilo.  I don't think we guarantee unreleased migrations.20:51
redrobotdave-mccowan are you building on my CR?20:51
redrobotdave-mccowan I had to rebase to include the openssl changes from jaosorior20:52
elmikoredrobot: ok, with that said. i'll look more at "the hard way" ;)20:52
aleedave-mccowan, redrobot so is there a defined set of patches I need to apply (in order)?20:52
redrobotdave-mccowan alee  I've been working with the assumption that others will be depending on my CR20:52
dave-mccowanredrobot, the other four are stand-alone.  all future work i'm doing in your CR20:52
redrobotdave-mccowan I'm working on getting my CR to pass the gate, so you can work off of it, instead of sharing it20:53
aleedave-mccowan, so I can apply redrobot patch and then your others in any order?20:53
dave-mccowanalee, redrobot  you'd probably have merge conflicts, but functionally any order is OK.  hopefully we can merge the other 4 today.  they're all small.20:54
dave-mccowanredrobot, i've just been working in your CR, since it needs help to pass the gate.  is that the way to do it?  or is there a more correct way?20:55
woodster_elmiko, is the 'hard way' having try/excepts around blocks that might fail in mysql but pass in postgres?20:56
redrobotdave-mccowan we're starting to step on each other's toes.  Your last patch erased a bunch of stuff I had done.20:56
elmikowoodster_: that or checking to see if the columns exist, but yea20:56
elmikowoodster_: it's doable20:56
redrobotdave-mccowan I'm -> <- this close to having a passing gate20:56
redrobotdave-mccowan as soon as I push the next patch you can rebase your work on top of it and send it as a dependent cr20:57
elmikowoodster_: then the next part will be figuring out what to do going forward20:57
dave-mccowanredrobot, in that case we're either duplicating effort or coding to a different design. :-)20:57
woodster_redrobot, that looks like a PTL trying to squeeze something out20:57
dave-mccowanredrobot, OK.  i'll wait for your patch, and then rebase.20:58
woodster_elmiko: are you using the db migration tool for this work? You can easily move between versions that way20:58
elmikowoodster_: i am, but i have errors when creating the fresh juno db20:59
elmiko(with the migration tool, that is)21:00
woodster_elmiko: hmm, I recall we were going to add a time-zero migration, that forklifts all the tables. Are you just loading the juno release though and then syncing that to a fresh database?21:00
elmikowoodster_: yea21:00
elmikowoodster_: although now that you suggest it, i could just roll back and forth between the good and bad migrations with the tool?21:01
woodster_elmiko: and just doing that sync fails for mysql21:01
elmikowoodster_: here's my workflow21:01
elmiko1. loadup stable/juno and let barbican auto create the db21:02
woodster_elmiko, forgot the ? there.  Yes, once the db is stood up at a given version, you can use the tool to upgrade and downgrade between versions (or blocks of versions)21:02
elmiko2. switch to master and run the db-migrate to head21:02
elmikowoodster_: should i be able to run the db-migrate on a fresh db?21:02
woodster_elmiko: yep 1 and 2 look good, but if you are testing a fix to an individual version, you could upgrade/downgrade -v <version before one I'm fixing>21:04
woodster_elmiko: and then upgrade -v <version to test> once you've updated that script file21:04
elmikowoodster_: cool! i'll give that a try21:05
woodster_elmiko: so upgrade if you are currently below the version to fix, or downgrade if above it21:05
elmikowoodster_: what if the upgrade fails, does it leave the db in an unrecoverable state?21:05
woodster_elmiko: but to stand a fresh db up with alembic alone would require a version file that has all the things in it21:05
elmikowoodster_: yea, thought so21:06
woodster_elmiko: if it fails the database is (should be) left in the current version21:06
elmikowoodster_: ack, thanks =)21:06
*** joesavak has quit IRC21:10
woodster_elmiko: I figure once the dust settles on what you are doing, this page can be updated with guidance for future devs: http://docs.openstack.org/developer/barbican/contribute/database_migrations.html21:11
elmikowoodster_: lol, that's a mouthful ;)21:11
woodster_elmiko: I also added a liberty note to check up on the grenade process...we really need to get migration gates working to keep us out of this mess21:11
elmikowoodster_: yea, it would also be nice to make the barbican-db-migrate work off of the installed version rather than adding the current path21:12
elmikowoodster_: in sahara we have migrations tested, maybe i can read up on some of that21:12
*** stanzi has joined #openstack-barbican21:13
woodster_elmiko: grenade is the project used for migrations, but projects have been having to check project specific stuff into it and tempest to make the gate work. Both tempest and grenade are moving away from that approach to instead call back into projects to configure and run tests, but that isn't supposed to be ready until the Liberty time frame.21:13
woodster_elmiko: do you mean having to specify the db URL as an argument vs getting it from teh installed config file?21:14
elmikowoodster_: ack on grenade. i can see value in specifying the dburl, i meant more that it looks like db-migrate wants to grab the migrations from the local directory rather than from the installed location.21:16
elmikowoodster_: but having it use the barbican-api.conf db connection would be a nice convenience feature21:17
openstackgerritDouglas Mendizábal proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST  https://review.openstack.org/17339621:20
*** alee has quit IRC21:21
*** stanzi has quit IRC21:21
redrobotdave-mccowan the more unit tests I fix, the more that fail >_>21:22
*** stanzi has joined #openstack-barbican21:22
dave-mccowanredrobot, i can relate.  are there still more failing in patch #5?21:22
redrobotdave-mccowan yeah.  I moved the keys from functional -> unit tests, so we can use just one source of keys21:23
redrobotdave-mccowan updated all the values for secret normalization21:23
redrobotdave-mccowan going to look at denormailize now21:23
redrobotdave-mccowan most of the failures I'm seeing now are because of the removed get_pem_components helper function21:24
dave-mccowanredrobot, i think that turns out to be helpful, since it points at the code that needs to be fixed.21:25
dave-mccowanredrobot, the code should now be PEM agnostic, so if it's trying to PEMify or de-PEMify something, it's probably wrong.21:26
redrobotdave-mccowan I'll tell you what, if you want to help out on this same CR again, pull down patch #5 and focus on the barbican/tests/tasks/test_certificate_resources.py ?21:26
dave-mccowanredrobot deal.  i'll that that one and tests/common/test_validators.py and common/validators.py21:27
*** dimtruck is now known as zz_dimtruck21:28
redrobotdave-mccowan cool, let me know if you need to edit anything in test_translations21:28
redrobotif either of us pushes up a patch, the other one will have to stash/pull down new patch/unstash21:28
dave-mccowanredrobot, OK.  i'll ignore translations.py   let's try to work only in separate files to help prevent conflicts.21:29
*** paul_glass has quit IRC21:35
*** stanzi has quit IRC21:36
*** stanzi has joined #openstack-barbican21:37
*** stanzi has quit IRC21:41
*** xaeth is now known as xaeth_afk21:51
openstackgerritDouglas Mendizábal proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST  https://review.openstack.org/17339621:53
redrobotdave-mccowan done with translations.  You'll want to stash / pull down patch # 6 / unstash21:53
redrobotdave-mccowan going to look at StoreCrypto next21:54
*** nkinder has quit IRC21:57
*** zz_dimtruck is now known as dimtruck22:02
openstackgerritDave McCowan proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST  https://review.openstack.org/17339622:02
redrobotdave-mccowan heh... looks like you didn't pull down my changes for test_translations.py22:04
dave-mccowanredrobot. :-(22:05
redrobotdave-mccowan no worries.  let me port them over to a new patch22:06
redrobotdave-mccowan maybe we should work on github instead22:07
redrobotdave-mccowan instead of ammending commits we can just start a branch22:07
redrobotand then squash it when we're done22:07
dave-mccowanredrobot, probably should have. but, we're almost done now.  i'll remember to squash next time.  looks like unit tests are about covered.  i'll look at functional.22:09
redrobotdave-mccowan functional is passing in my box22:10
dave-mccowanredrobot.  sweet.  ship it!22:10
redrobotdave-mccowan 2 seconds on patch # 822:11
openstackgerritDouglas Mendizábal proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST  https://review.openstack.org/17339622:15
redrobotdave-mccowan ok, pull down # 8 and you can look at the functional22:15
redrobotdave-mccowan I'll finish up the unit tests22:15
redrobotdave-mccowan I think you may have uncommented more functional tests22:16
dave-mccowanredrobot, i think if we merge with my other pending CRs, we'll have all 18 passing now.22:17
redrobotdave-mccowan ok, let's get your CRs merged, and then rebase this big one on top of those.22:18
*** alee has joined #openstack-barbican22:18
redrobotdave-mccowan I like small CRs.   All reviewed up.  Maybe alee can take a look at them as well22:25
*** stanzi has joined #openstack-barbican22:25
dave-mccowanredrobot, i have some fixes to test_certificate_order.py coming soon to our joint CR22:26
*** igueths has quit IRC22:26
redrobotdave-mccowan just be sure to check for new patches before you --amend22:26
dave-mccowanredrobot, to fetch the latest (after stash): git fetch? git review -d?22:29
redrobotgit review -d22:29
redrobotdave-mccowan git review -d 17339622:29
redrobotso is it all plugins that pass base64(payload) or just secret store?22:31
openstackgerritDave McCowan proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST  https://review.openstack.org/17339622:32
*** tkelsey has joined #openstack-barbican22:34
aleeredrobot, dave-mccowan -- looking22:36
*** tkelsey has quit IRC22:38
elmikowoodster_: ok, i've got something working for both mysql and postgresql22:41
elmikowoodster_: needs a little more cleanup, but it's a start22:42
redrobotelmiko w00t!22:42
elmikoredrobot: when i get this set, i was thinking about making a bug and then posting the review against it. sound good?22:43
redrobotelmiko I think this is the bug for it? https://bugs.launchpad.net/barbican/+bug/133690122:44
openstackLaunchpad bug 1336901 in Barbican "alembic migration[mysql] fails with kek_data migration" [Low,Confirmed]22:44
*** stanzi has quit IRC22:45
*** stanzi has joined #openstack-barbican22:45
elmikoredrobot: haven't hit that one yet, but i'll get to it22:48
elmikoredrobot: there are actually several migrations that need adjustment22:48
*** stanzi has quit IRC22:50
*** dave-mccowan has quit IRC22:54
woodster_elmiko: redrobot yeah that is an old bug, from before Juno.22:59
*** nkinder has joined #openstack-barbican23:01
*** dave-mccowan has joined #openstack-barbican23:10
*** chlong has joined #openstack-barbican23:10
*** kebray has quit IRC23:11
elmikowoodster_: ack23:11
*** dave-mccowan has quit IRC23:15
*** dave-mccowan has joined #openstack-barbican23:19
woodster_elmiko: so yeah, we are only working on migrations after at/after Juno then23:22
elmikowoodster_: ok, cool23:24
*** dimtruck is now known as zz_dimtruck23:24
elmikoi'm basically just trying to update a juno db to current master23:24
*** zz_dimtruck is now known as dimtruck23:26
*** crc32 has quit IRC23:39
dave-mccowanredrobot, those four patches still need workflow, so they can merge and then i can do the rebase.23:41
*** chlong has quit IRC23:43
*** dimtruck is now known as zz_dimtruck23:45
*** chlong has joined #openstack-barbican23:45
openstackgerritDave McCowan proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST  https://review.openstack.org/17339623:49
alee_afkdave-mccowan, ping23:49
dave-mccowanalee_afk pong23:49
alee_afkdave-mccowan, so looking at your patches ..23:50
*** zz_dimtruck is now known as dimtruck23:50
alee_afkdave-mccowan, https://review.openstack.org/#/c/173163/1/functionaltests/api/v1/smoke/test_rsa.py,cm23:50
alee_afkthe substatus "cert_request_pending"23:50
alee_afkis that the status that is set after the request is made to the cert plugin?23:50
dave-mccowanalee_afk, yes.  these tests pass when the rest of the code is fixed, so it must be. :-)23:51
alee_afkok -- I'll be looking at these carefully once these are in ..23:52
alee_afkI just wanted to make sure it is not the substatus set even before that point23:52
alee_afkie that its actually getting that far.23:52
dave-mccowanalee_afk here is what i get back.  http://ur1.ca/k6cpf23:53
alee_afkok good23:54
alee_afkdave-mccowan, ok -workflowed 3 of them.23:56
alee_afkcalled to dinner ..23:56
alee_afkdave-mccowan, redrobot - I'll apply and review https://review.openstack.org/173396 later tonight23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!