Tuesday, 2016-08-23

openstackgerritArun Kant proposed openstack/barbican-specs: Adding spec for supporting multiple secret store backends  https://review.openstack.org/26397200:05
*** edtubill has joined #openstack-barbican00:11
openstackgerritArun Kant proposed openstack/barbican: Adding API docs for multiple backend support changes.  https://review.openstack.org/34180300:31
openstackgerritArun Kant proposed openstack/barbican: Adding rest API for secret-stores resource (Part 4)  https://review.openstack.org/35816200:33
*** diazjf has joined #openstack-barbican01:00
*** jamielennox is now known as jamielennox|away01:11
*** jamielennox|away is now known as jamielennox01:16
*** su_zhang has quit IRC01:26
*** diazjf has quit IRC01:34
*** hockeynut has quit IRC01:36
*** dave-mccowan has joined #openstack-barbican02:34
*** jamielennox is now known as jamielennox|away02:49
*** dave-mccowan has quit IRC02:56
*** dave-mccowan has joined #openstack-barbican02:59
*** jamielennox|away is now known as jamielennox03:06
*** dave-mccowan has quit IRC03:21
*** diazjf has joined #openstack-barbican03:24
*** diazjf has quit IRC03:32
*** su_zhang has joined #openstack-barbican04:39
*** jamielennox is now known as jamielennox|away04:49
*** edtubill has quit IRC04:50
*** jaosorior has joined #openstack-barbican05:10
*** jamielennox|away is now known as jamielennox05:23
*** su_zhang has quit IRC05:26
*** su_zhang has joined #openstack-barbican05:26
*** jamielennox is now known as jamielennox|away06:11
*** pcaruana has joined #openstack-barbican06:14
*** woodster_ has quit IRC06:19
*** andreas_s has joined #openstack-barbican06:43
*** su_zhang has quit IRC06:48
*** su_zhang has joined #openstack-barbican06:48
*** su_zhang has quit IRC06:52
*** f13o has quit IRC06:53
*** shohel has joined #openstack-barbican07:13
openstackgerritgecong proposed openstack/barbican: Remove white space between print and ()  https://review.openstack.org/35901307:50
*** openstackgerrit has quit IRC08:03
*** openstackgerrit has joined #openstack-barbican08:04
*** f13o has joined #openstack-barbican08:07
*** f13o has quit IRC08:31
*** lixiaoy1 has joined #openstack-barbican09:36
*** shohel has quit IRC10:07
*** shohel has joined #openstack-barbican10:08
*** ntpttr has quit IRC10:30
*** ntpttr has joined #openstack-barbican10:35
*** shohel has quit IRC11:30
*** shohel has joined #openstack-barbican11:30
*** jaosorior has quit IRC11:50
*** jaosorior has joined #openstack-barbican11:51
*** phschwartz has quit IRC12:14
*** nkinder has joined #openstack-barbican12:31
*** phschwartz has joined #openstack-barbican12:38
*** alee has quit IRC12:39
*** dave-mccowan has joined #openstack-barbican12:45
*** chlong has quit IRC12:56
*** woodster_ has joined #openstack-barbican12:56
*** chlong has joined #openstack-barbican13:13
*** alee has joined #openstack-barbican13:29
*** dmsimard has joined #openstack-barbican13:51
dmsimardHi #openstack-barbican o/ A recent commit in Cinder seems to have broken encrypted volumes with Barbican. I'm not very familiar with this and would appreciate if someone could help. We've filed a bug about it https://bugs.launchpad.net/cinder/+bug/161505913:52
openstackLaunchpad bug 1615059 in Cinder "Cinder fails to create a crypted volume when barbican is used as key manager" [Undecided,In progress] - Assigned to Lisa Li (lisali)13:52
dmsimardThere was a review that attempts to fix the issue but it uncovers another problem -- I guess encrypted volumes through barbican as a key manager is largely untested in Cinder13:53
woodster_dmsimard: kfarr is the most knowledgeable about the encrypted workflows. alee did you ever get cinder workflows working in your setup?13:56
aleewoodster_, I did indeed.  in fact it appears that its through my setup (RDO CI) that this issue was uncovered ..13:58
dmsimardalee: yeah, it was, actually :)13:59
dmsimardalee: RDO CI typically runs ahead of puppet-openstack-integration, we pull their tests and run them outside of the gate14:00
dmsimardalee: but the issue also reproduced upstream in https://review.openstack.org/#/c/357645/14:00
aleedmsimard, OK will investigate14:02
*** pcaruana has quit IRC14:02
dmsimardalee: thanks, appreciate it.14:04
dmsimardThe issue is bound to reproduce in Ubuntu as well, they just don't update their packages as often :)14:05
*** zz_dimtruck is now known as dimtruck14:17
*** pcaruana has joined #openstack-barbican14:17
*** edtubill has joined #openstack-barbican14:17
*** spotz_zzz is now known as spotz14:29
*** jaosorior is now known as jaosorior_away14:32
*** randallburt has joined #openstack-barbican14:50
*** randallburt1 has joined #openstack-barbican14:52
*** randallburt has quit IRC14:55
*** shohel has quit IRC14:56
*** hockeynut has joined #openstack-barbican14:58
*** edtubill has quit IRC15:07
*** andreas_s has quit IRC15:25
aleedmsimard, ping15:38
aleedmsimard, the reason you get a failure now is because something is wonky with the barbican config15:39
aleedmsimard, for some reason, no secret_store plugins are enabled15:39
aleedmsimard, you guys are running whats in puppet-openstack-integration ?15:40
*** pcaruana has quit IRC15:56
woodster_alee: Only the initial attempt to load a plugin gives you the root cause exception to the load. I think we need to save the initial root cause exception when plugins are loaded by stevedore, and then re-output that stacktrace for plugin load attempts after that.15:59
*** jaosorior_away is now known as jaosorior16:00
*** michauds has joined #openstack-barbican16:08
*** pcaruana has joined #openstack-barbican16:09
*** nkinder has quit IRC16:11
*** jaosorior has quit IRC16:18
dmsimardalee: yeah.16:36
dmsimardalee: we're running puppet-openstack-integration but outside the gate16:36
dmsimardalee: although I guess it also reproduces in the gate as per the review I sent you16:36
*** nkinder has joined #openstack-barbican17:05
aleedmsimard, its pretty weird that the barbican config is messed up.17:07
dmsimardalee: not going to disagree17:08
aleedmsimard, ok - I think the basic fix is to be explicit in which plugins will be enabled17:19
aleedmsimard, this will need to be a change in openstack-puppet-integration17:19
aleedmsimard, I can put up a change, and you can link to it?17:20
dmsimardalee: there's a couple layers involved, I won't be able to exactly test both your change and the cinder "fix" simultaneously17:21
dmsimardif the cinder fix looks good to you, we can pressure them to merge it17:22
dmsimardand then once we build a package with that review, we can rebase https://review.openstack.org/#/c/357645/ with an up-to-date repo on top of your puppet-openstack-integration fix17:22
aleedmsimard, well - kfarr is the exper t here on the cinder side - but it looks good to me so far17:23
dmsimardalee: kfarr actually -1'd it https://review.openstack.org/#/c/358670/17:23
aleedmsimard, well - I'd defer to her -- I dont know this code ..  she seemed to have an objecttion to a change not done in the mock key managger test17:25
dmsimardand the contributor is in APAC /me sighs17:26
aleedmsimard, so most likely the change is ok except for incorrect mock key manager test17:26
aleedmsimard, anyways I'll put my change up17:26
dmsimardalee: is it forward compatible ?17:27
aleeshould be .. its just makiong explicit what was not before17:27
*** su_zhang has joined #openstack-barbican17:29
*** hockeynut has quit IRC17:33
aleedmsimard, https://review.openstack.org/35936317:36
dmsimardalee: ack, ty17:37
arunkantalee, can you re-review multi-backend spec ( https://review.openstack.org/#/c/263972/) and API docs review..change is in API response only as per redrobot comments17:38
aleearunkant, will do17:39
arunkantredrobot: can you review API docs (https://review.openstack.org/#/c/341803) . I was hoping if these 2 can be merged by this week.17:39
aleewoodster_, ping17:51
*** su_zhang has quit IRC17:51
*** su_zhang has joined #openstack-barbican17:51
aleewoodster_, redrobot I'm trying to figure out whats going on with plugin loads ..17:52
aleewoodster_, redrobot if I look in secret_store.py , I see default plugins being defined ..17:52
*** zhugaoxiao has joined #openstack-barbican17:53
aleewoodster_, redrobot but when I look at store_crypto.py, I do not see a default plugin there ..17:55
*** Administrator__ has quit IRC17:55
woodster_alee: The repo is self contained with the default insecure plugin17:55
aleewoodster_, redrobot - should there be?17:55
aleewoodster_, redrobot but is it?17:56
aleewoodster_, redrobot that is -- where is the definition of a default store_crypto plugin?17:57
woodster_alee: looking now....17:57
woodster_alee: this is the one: https://github.com/openstack/barbican/blob/master/barbican/plugin/crypto/simple_crypto.py17:58
* woodster_ default one used that is17:58
*** dimtruck is now known as zz_dimtruck17:59
aleewoodster_, right - but there is no default setting for enabled_crypto_plugin17:59
aleewoodster_, unless its here --? barbican/barbican/plugin/crypto/manager.py ?18:00
*** su_zhang has quit IRC18:00
*** su_zhang has joined #openstack-barbican18:01
*** zz_dimtruck is now known as dimtruck18:01
*** jamielennox|away is now known as jamielennox18:01
*** su_zhang has quit IRC18:01
*** su_zhang has joined #openstack-barbican18:02
*** su_zhang has quit IRC18:02
*** su_zhang has joined #openstack-barbican18:02
woodster_alee: So this configures the plugin: https://github.com/openstack/barbican/blob/master/etc/barbican/barbican.conf#L26018:05
woodster_alee: and this defines the possible choices: https://github.com/openstack/barbican/blob/master/setup.cfg#L4218:06
aleewoodster_, so lets imagine that barbican.conf has no value for enabled_secretstore_plugins or enabled_crypto_plugins18:07
aleewoodster_, what happens then?18:07
woodster_alee: well that could be a problem :)18:07
aleewoodster_, would it?  or should it be?18:08
aleewoodster_, https://github.com/openstack/barbican/blob/master/barbican/plugin/crypto/manager.py#L3218:08
aleewoodster_, https://github.com/openstack/barbican/blob/master/barbican/plugin/interface/secret_store.py#L3318:09
aleewoodster_, the way that works -- shouldn't the default plugin be the simple_crypto one?18:10
woodster_alee: well, I know the nova team has moved to a no-config file needed approach18:11
aleewoodster_, right - the idea being presumably that it just works out of the box18:12
woodster_alee: yeah that should load the things.  So what specific error are you seeing?18:12
*** hockeynut has joined #openstack-barbican18:13
aleewoodster_, no plugin found :/18:13
aleewoodster_, getting link18:13
woodster_alee: you have to look at the stack trace the first time no plug founds is seen after boot up18:13
aleewoodster_, here is the config file
woodster_alee: I can't hit that host...can you pastebin it?18:14
aleewoodster_, log file ..18:14
aleewoodster_, really .. didn't think it was internal ...18:15
aleewoodster_, ok - just a sec18:15
aleewoodster_, http://paste.ophttp://paste.openstack.org/show/562530/18:16
aleewoodster_, config http://paste.openstack.org/show/56253118:17
woodster_alee: it is not able to find a plugin to support the type order....we probably should make that error message more explicit, like "Could not find a secret storage backend to support generating a secret of type 'xyz'"18:21
woodster_alee: well, a 'crypto backend' rather than 'secret storage backend'18:21
aleewoodster_, right - but the simple crypto plugin does support generating a secret18:22
dmsimardalee: that host isn't internal18:23
dmsimardalee: it's a public swift cluster18:23
dmsimardwoodster_: ^18:23
woodster_dmsimard: oh that makes sense18:24
woodster_alee: the simple crypto does support some generation: https://github.com/openstack/barbican/blob/master/barbican/plugin/crypto/simple_crypto.py#L20218:24
woodster_it would help if we logged out what the order info was when that failure occurs18:25
woodster_so basically whatever generation is being request is failing that above method most likely18:25
aleewoodster_, we can get that from the cinder-api logs here .. just a sec ..18:26
aleewoodster_, if you can get to that -- http://logs.openstack.org/45/357645/4/check/gate-puppet-openstack-integration-3-scenario002-tempest-centos-7/3273ca7/logs/cinder/cinder-api.txt.gz#_2016-08-22_08_01_11_49218:27
aleewoodster_, sorry not the rright one ..18:27
woodster_alee: oh cool, I just saw cinder key errors in there :)18:29
* woodster_ didn't know they were using task flow now18:29
aleewoodster_, paste.openstack.org/show/562532/18:29
woodster_alee: 512 aes is not supported: https://github.com/openstack/barbican/blob/master/barbican/plugin/crypto/crypto.py#L5418:31
aleewoodster_, yes indeed :)18:31
aleedmsimard, ^^18:32
woodster_so better logging would have made that 10x more obvious at least.18:32
aleewoodster_, yeah18:32
dmsimardso cinder is passing a keylength that is too high for what is supported by that plugin ?18:32
woodster_alee: do you have any cycles to put up a CR to improve the exception message?18:32
aleewoodster_, yeah - and I'll up the plugin too18:33
woodster_dmsimard: I don't think the plugin would have a problem with 512, but our 'is supports 512' checker is rejecting it18:33
dmsimardwoodster_: fair enough18:33
aleedmsimard, there is no reason the plugin should not support that18:33
dmsimardalee: are we the ones passing that 512 value through puppet ?18:33
dmsimardalee: looks like not, a default from cinder ?18:34
woodster_alee: dmsimard what is the largest aes key size folks are typically generating nowadays? That upper limit should be at least 1024 I'm thinking18:34
aleedmsimard, likely yes -- they probably uppped it18:34
dmsimardwoodster_: I have no clue tbh :p18:35
*** diazjf has joined #openstack-barbican18:36
dmsimardalee: looks like it's defined here https://github.com/openstack/cinder/blob/3ad7384913546a71b32b7e321c035183eedfc255/cinder/volume/flows/api/create_volume.py#L36418:36
dmsimardgoing upwards to try and figure out where that's from18:36
aleedmsimard, oh thats probably in the tempest test18:37
dmsimardhttp://docs.openstack.org/mitaka/config-reference/block-storage/volume-encryption.html mentions 512 in the example tests18:37
woodster_dmsimard: well that error message is a fail for sure...creating a LP bug for that now...18:37
dmsimardalee: oh, right, tempest probably creates a volume type with aes 51218:38
dmsimardand that's where it fails18:38
aleedmsimard, https://github.com/openstack/tempest/blob/master/tempest/scenario/test_encrypted_cinder_volumes.py#L5618:38
dmsimardso, two solutions here -- 1) bump the plugin max (what are the impacts?) 2) adapt the tempest test18:39
dmsimardalee: so, just making sure18:40
dmsimardalee: that's the second issue, right ? The second problem I brought up 618:40
aleedmsimard, interesting - that value has been there for awhile now .. guess it never was really tested with barbcian before :/18:40
dmsimardalee: cinder doesn't test barbican encrypted volumes upstream18:41
dmsimardthis is all RDO CI :D18:41
*** su_zhang has quit IRC18:41
aleedmsimard, glad Emilien made me put it in openstack-puppet-integration :)18:41
*** su_zhang has joined #openstack-barbican18:41
woodster_dmsimard: alee I'd prefer to add 512, 1024, 2048, 4096 as options (assuming they all actually work without 'sploding)18:41
aleewoodster_, +118:42
dmsimardalee: /me nods p-o-i has typically better coverage than devstack18:42
aleewoodster_, 4096 symmetric key length?18:44
aleewoodster_, not sure it makes sense to go beyond 1024 ..18:44
woodster_alee: sure, why not? :) We'll be a few years ahead of the crowd18:44
dmsimardgoogling a bit, it doesn't look like 4096 aes is even a thing18:45
woodster_alee: well at least to 1024 I think18:45
aleewoodster_, ok - I'll up to 102418:45
aleedmsimard, right - - 4096 and for asymm keys ..18:46
dmsimard4096 for RSA maybe18:46
woodster_dmsimard: asymmetric (PKI) is already maxed at 409618:48
dmsimardwoodster_: did you file a bug for improved exception handling + plugin key length bump ?18:48
dmsimardwant to reference it on our end18:49
aleewoodster_, https://www.keylength.com/en/4/18:50
dmsimardalee: so aes-512 isn't even a thing ?18:51
woodster_dmsimard: alee this is to focus on the error message: https://bugs.launchpad.net/barbican/+bug/161617918:51
openstackLaunchpad bug 1616179 in Barbican "Error message too vague for no supporting crypto plugin found for secret generation" [Undecided,New]18:51
woodster_dmsimard: alee I'll submit one for the max aes size18:52
dmsimardwoodster_: thanks I'll wait18:52
aleewoodster_, http://crypto.stackexchange.com/questions/20253/why-we-cant-implement-aes-512-key-size18:52
aleewoodster_, I'm not sure aes 512 is even a thing ..18:52
*** su_zhang has quit IRC18:52
woodster_alee: oh good point!18:53
dmsimardalee: so the fix is in tempest then18:53
woodster_well, I did say to boost the size only if it actually worked :)18:53
dmsimardalee: we can argue that testing with 256 in tempest is sufficient18:53
aleewoodster_, https://www.researchgate.net/publication/220793242_AES-512_512-Bit_Advanced_Encryption_Standard_algorithm_design_and_evaluation18:54
dmsimardhowever, what's up with those docs http://docs.openstack.org/mitaka/config-reference/block-storage/volume-encryption.html that mention aes-51218:54
dmsimardthe docs mention aes-512 since juno at least18:54
aleedmsimard, well - the same folks that wrote the tempest test wrote the docs18:55
dmsimardlol, you think ?18:55
dmsimardmaybe aes-512 is some super secret nsa stuff18:55
woodster_well I saw a research paper on aes-512, but even NIST isn't testing that yet (see here...just 256 max aes mentioned: http://csrc.nist.gov/groups/STM/cavp/documents/aes/aesval.html)18:58
woodster_maybe this is no accident?: https://en.wikipedia.org/wiki/Tempest_(codename)18:59
* woodster_ the naming of the test framework that is19:00
aleewoodster_, I'd be curious if kfarr could provide some feedback here ..19:01
woodster_alee: she might, but then she'd have to disappear you ;)19:06
dmsimardalee: I'm asking in #openstack-cinder19:08
dmsimardalee: oh but hey, kfarr is actually the one that submitted https://review.openstack.org/#/c/251503 which broke everything T_T19:09
dmsimardSee the commit message19:09
dmsimard"Another fix will be needed to address the use case of aes-xts with a key size of 512 -- key managers may not be able to create 512 bit AES keys."19:09
dmsimard"Another fix will be needed to address the use case of aes-xts with a key size of 512 -- key managers may not be able to create 512 bit AES keys."19:10
dmsimarder, wrong channel19:11
aleedmsimard, woodster_  ok - I think we need to wait to talk with kfarr19:11
aleeand figure out what she's trying to do19:11
dmsimardalee: so I guess aes-xts != aes ?19:11
aleedmsimard, yeah -- or simple plugin doesn19:11
aleedoes not do much to distinguish19:12
dmsimardalee: so I'd like to file a bug about this so we don't lose track of it. Should I use https://bugs.launchpad.net/cinder/+bug/1514546 which is the bug referenced in kfarr's commit ?19:13
openstackLaunchpad bug 1514546 in Cinder "Cinder volume encryption uses default parameters for keys" [Undecided,Fix released] - Assigned to Lisa Li (lisali)19:13
aleedmsimard, sure19:13
*** pcaruana has quit IRC19:20
*** hockeynut has quit IRC19:28
*** kfarr has joined #openstack-barbican19:30
*** su_zhang has joined #openstack-barbican19:41
*** su_zhang has quit IRC19:46
openstackgerritArun Kant proposed openstack/barbican: Adding rest API for secret-stores resource (Part 4)  https://review.openstack.org/35816219:51
*** kfarr has quit IRC19:55
*** kfarr has joined #openstack-barbican19:57
*** su_zhang has joined #openstack-barbican20:08
openstackgerritMax Abidi proposed openstack/python-barbicanclient: Validate key order meta fields.  https://review.openstack.org/32010020:09
*** dmsimard has left #openstack-barbican20:09
openstackgerritMerged openstack/barbican-specs: Adding spec for supporting multiple secret store backends  https://review.openstack.org/26397220:14
*** haplo37__ has joined #openstack-barbican20:14
*** hockeynut has joined #openstack-barbican20:27
*** sigmavirus is now known as sigmavirus|away20:29
*** gyee has joined #openstack-barbican20:37
*** diazjf has quit IRC20:38
*** nkinder has quit IRC20:40
*** diazjf has joined #openstack-barbican20:59
*** su_zhang has quit IRC20:59
*** su_zhang_ has joined #openstack-barbican21:01
woodster_alee: That CR from kfarr uses 256 bit keys with 'aes'...are you thinking that is what is causing issues though?21:05
aleewoodster_, eh?21:06
aleewoodster_, the issue was that the tempest test was asking for aes 51221:06
woodster_alee: just trying to catchup on that 512 bit issue...are you thinking this CR is introducing that bug?: https://review.openstack.org/#/c/251503/21:06
woodster_alee: oh got it, so not related to that CR then21:07
aleewoodster_, it is related in the sense that before this fix, cinder was not passing parameters for bit size21:07
aleeand so it was taking the default21:07
aleewhich was aes 25621:08
woodster_alee: ah got it, ok that makes sense then21:08
aleewoodster_, now that kfarr fixed the cinder code to actually pass through what was asked for ..21:08
aleewoodster_, we realized what was asked for was bogus21:08
woodster_alee: yep, now it's revealing the problem that was masked before21:09
kfarralee, woodster_ there's a bunch of underlying problems D:21:09
aleeyes - not the least of which is that we dont test encryption with barbicaqn upstream21:10
woodster_kfarr: none with barbican at least? :)   Other than cryptic crypto logging?21:10
aleecryptic crypto indeed21:10
kfarrwoodster_ ah, no, not barbican problems.21:10
woodster_alee: you mean with real backends? Or integrated with other projects?21:10
aleeintegrated with other projects21:11
aleethe only reason we found this was because RDO CI now runs the volume encryption test with barbican21:11
woodster_alee: so less than 2% adoption rate on integration testing?21:11
* woodster_ I did say that with a straight face21:13
*** shohel has joined #openstack-barbican21:14
*** martial_ has joined #openstack-barbican21:14
*** dimtruck is now known as zz_dimtruck21:21
*** zz_dimtruck is now known as dimtruck21:23
*** martial_ has left #openstack-barbican21:23
*** hockeynut has quit IRC21:31
*** su_zhang_ has quit IRC21:42
*** su_zhang has joined #openstack-barbican21:42
*** diazjf has quit IRC21:52
*** diazjf has joined #openstack-barbican21:54
openstackgerritOpenStack Proposal Bot proposed openstack/barbican: Updated from global requirements  https://review.openstack.org/35231522:15
openstackgerritOpenStack Proposal Bot proposed openstack/castellan: Updated from global requirements  https://review.openstack.org/35231622:15
*** alee has quit IRC22:16
*** spotz is now known as spotz_zzz22:17
*** diazjf has quit IRC22:18
*** shohel has quit IRC22:19
*** dimtruck is now known as zz_dimtruck22:30
*** michauds has quit IRC22:36
*** alee has joined #openstack-barbican23:02
*** chlong has quit IRC23:03
*** kfarr has quit IRC23:19
*** randallburt1 has quit IRC23:44

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!