Wednesday, 2018-01-10

openstackgerritTim Burke proposed openstack/swift master: Add pipeline modification test for previously-recommended pipelines  https://review.openstack.org/53236000:00
timburkewhoops -- included a swiftstack tag accidentally00:01
openstackgerritTim Burke proposed openstack/swift master: Add pipeline modification test for previously-recommended pipelines  https://review.openstack.org/53236000:10
timburke...which in turn caused me to think there was an extra change in pipelines...00:10
*** kei_yama has quit IRC00:11
*** JimCheung has quit IRC00:19
*** JimCheung has joined #openstack-swift00:20
*** vint_bra has joined #openstack-swift00:25
openstackgerritSamuel Merritt proposed openstack/swift master: proxy: make the right number of container updates  https://review.openstack.org/53152700:34
openstackgerritSamuel Merritt proposed openstack/swift master: Limit object-expirer queue updates on object DELETE, PUT, POST  https://review.openstack.org/53234200:34
*** andymccr has quit IRC00:36
*** andymccr has joined #openstack-swift00:40
notmynamekota_: would you have a chance to look at https://review.openstack.org/#/c/365371/ today (or soon)?00:42
patchbotpatch 365371 - swift - Add support for data segments to SLO and Segmented...00:42
*** corvus has quit IRC00:44
*** tovin07_ has joined #openstack-swift00:44
*** andymccr has quit IRC00:51
*** JimCheung has quit IRC00:54
*** JimCheung has joined #openstack-swift00:54
*** andymccr has joined #openstack-swift00:55
*** JimCheung has quit IRC00:58
*** awkwardpenguin has joined #openstack-swift01:07
*** awkwardpenguin has quit IRC01:11
kota_notmyname: will do today01:13
*** fishbone_ has joined #openstack-swift01:24
*** JimCheung has joined #openstack-swift01:26
*** JimCheung has quit IRC01:30
fishbone_hello all, can anyone help me, I'm receiving an error on the proxy server Unable to validate token: Identity server rejected authorization necessary to fetch token data01:36
fishbone_Identity response: {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}01:36
fishbone_X.X.X.X X.X.X.X 10/Jan/2018/01/33/38 HEAD /v1/AUTH_75e4b7c8c5b8484daf7da7b249dc53aa HTTP/1.0 50301:37
fishbone_the X ip address is incorrect, I do not see a directive for setting the proxy server IP01:37
fishbone_I think that's where my issue lies01:37
fishbone_I did set the bind host directive but it doesn't change this error01:37
*** d0ugal has quit IRC01:43
*** m_kazuhiro has joined #openstack-swift01:50
m_kazuhirogood morning01:50
mattoliveraum_kazuhiro: morning01:53
mattoliveraufishbone_: what authentication are you using? keystone? if so check your keystone log, looks like an auth problem.01:54
m_kazuhiromattoliverau: morning! Thank you for your +2 to my internal client patch.01:54
mattoliveraum_kazuhiro: no problem. plan to look closer at the general task queue today. But been caught up in non swift stuff this morning at work.01:54
m_kazuhiromattoliverau: ok!01:55
*** two_tired has joined #openstack-swift01:58
fishbone_thanks +mattoliverau for your reply the error i get is Authorization failed. The request you have made requires authentication. from 192.168.1.31: Unauthorized: The request you have made requires authentication.02:01
fishbone_I wonder if my proxy server auth settings are icorrect02:01
*** d0ugal has joined #openstack-swift02:01
mattoliveraufishbone_: could be. hard to say. Are you using keystone for authentication?02:02
fishbone_+mattoliverau: I followed documentation, do I need to specify the auth version in the conf file?02:02
fishbone_+mattoliverau: yes it is keystone02:02
fishbone_that was the keystone log that pulled that last error02:02
*** two_tired has quit IRC02:02
*** vint_bra has quit IRC02:04
mattoliverauoh ok. Then you at least pointing to keystone. Is the user your using a member of any of the operator_roles you defined in the proxy.conf (in keystone).02:05
mattoliverau*proxy-server.conf02:05
mattoliveraulet me rephrase that. In the Swift proxy-server.conf you defined operator_roles that match to keystone roles. Is your user a member of one of those roles in keystone?02:08
fishbone_+mattoliverau: ahh I see, I will verify that right away02:09
fishbone_+mattoliverau: thanks a lot I overlooked adding the role, although now it's saying that it cannot find the project named service, which is there02:22
*** jeblair has joined #openstack-swift02:24
fishbone_+mattoliverau: my apologies, I cleared that error, now I am getting Account HEAD failed: http://controller:8080/v1/AUTH_75e4b7c8c5b8484daf7da7b249dc53aa 400 Bad Request02:28
*** two_tired has joined #openstack-swift02:32
*** two_tired has quit IRC02:36
*** two_tired has joined #openstack-swift02:38
mattoliveraufishbone_: have you set02:58
mattoliverauaccount_autocreate = true in swift-proxy.conf?02:58
fishbone_+mattoliverau: Yes that is set to true03:02
*** awkwardpenguin has joined #openstack-swift03:09
*** awkwardpenguin has quit IRC03:14
fishbone_+mattoliverau: might it be this? account-replicator: Skipping /dev/vdb as it is not mounted03:22
fishbone_I've been ignoring those errors as the drives are clearly mounted and acessible via the storage nodes03:23
fishbone_but I am getting those errors for all drives03:23
openstackgerritSamuel Merritt proposed openstack/swift master: Limit object-expirer queue updates on object DELETE, PUT, POST  https://review.openstack.org/53234203:29
openstackgerritSamuel Merritt proposed openstack/swift master: Don't make async_pendings during object expiration  https://review.openstack.org/53238303:29
*** jistr has quit IRC03:31
mattoliveraufishbone_: ok, any errors in your account-server log?03:38
fishbone_+mattoliverau I didn't know there was one, I've been using syslog, where would I find that log?03:39
mattoliveraufishbone_: depends where you set it up. if the proxy is syslogging so shouhld the account-server.03:39
mattoliveraualso, it's you said its skipping /dev/vdb.. that sounds like your pointing to actauly devices, not a devices mount point in your rings.03:40
*** jeblair has joined #openstack-swift03:40
mattoliverauSo you should mount all your devices somethere say, /mnt/vdb /mnt/vdc etc.. the devices mounted with xfs.03:41
mattoliverauthen your rings should point at these locations. ie. your devices in config files would (in this example) be /mnt and your devices in the ring could be named vdb (path from devices).03:42
mattoliveraualso make sure the swift user (or whatever user your running the swift daemons as) has access/permissions on these mount points03:43
mattoliveraufishbone_: ^03:43
fishbone_ah yes +mattoliverau, the only errors I see are the mount errors, I see the devices mounted in /srv/node as per documentation03:43
mattoliveraucool03:43
mattoliveraujust double checking :)03:44
fishbone_+mattolivera: thanks, I have a snippet of my errors here: http://paste.openstack.org/show/641850/03:44
fishbone_+mattolivera: I even tried remounting them and rebooting but those errors show, I ignored them because the mounts are accessible as far as I can see03:45
fishbone_+mattolivera, also yes they do have full permissions set03:52
mattoliveraufishbone_: any chance you can show me your account ring? paste.o.o the output of 'swift-ring-builder /etc/swift/account.builder03:53
mattoliverauor whatever the path to your account builder is03:53
fishbone_+mattoliverau sure http://paste.openstack.org/show/641865/ thanks for all of your help03:59
fishbone_+mattoliverau: I followed the openstack documentation on this and it mentioned to pull the gz files from the proxy and replicate them to the storage nodes, did I have to do that for the .builder files as well?04:00
mattoliveraufishbone_: no you don't have to. but when you edit your rings, you'll edit the builder files and then generate rings so back em up :)04:00
mattoliveraufishbone_: ok so you've pointed directly to the devices in the account ring. what is the 'devices' in your account-server.conf?04:01
fishbone_+mattoliverau: ahh, I see that now04:02
fishbone_In the conf it is /srv/node04:02
mattoliverauit seems your object ring is correct. so just a mistake there I suspect04:02
mattoliveraucool, do mount points I assume are vdb and xdc under /srv/node which means in your ring you just need to point to vdb vdc.04:03
fishbone_+mattoliverau, hmmm, it seems I pointed to /srv/node in the mount point itself and all config files, any reason why the ring would point to /dev/ ?04:04
mattoliverauit shouldn't04:04
mattoliverauunless you told it to when putting adding the device04:04
mattoliverauwhat does your object.builder say?04:04
fishbone_+mattoliverau, that one omits the /dev/ prefix, maybe I made a mistake creating the account.builder, I will try to recreate it04:06
fishbone_+mattoliverau: thanks for everything, I will let you know if this works out, it seems like it will04:07
mattoliveraufishbone_: please do and good luck. Have fun with Swift and if you are still stuck just ask :)04:08
mattoliveraufishbone_: oh, and which documentation where you using (in case creating the account ring is incorrect there)?04:09
fishbone_+mattoliverau: I m using from the opnestack website https://docs.openstack.org/swift/pike/install/initial-rings.html I checked my bash history, I definitely fat fingered this one the docs are solid04:10
mattoliverausweet, good to know :)04:11
*** jeblair has joined #openstack-swift04:12
fishbone_+mattoliverau: all works now, thanks again!04:19
mattoliveraufishbone_: awesome!04:22
*** jeblair has quit IRC04:24
*** JimCheung has joined #openstack-swift04:28
*** JimCheung has quit IRC04:32
*** m_kazuhiro_ has joined #openstack-swift04:33
*** fishbone_ has quit IRC04:36
*** m_kazuhiro has quit IRC04:36
*** links has joined #openstack-swift04:37
*** itlinux has quit IRC04:39
*** links has quit IRC04:40
*** links has joined #openstack-swift04:46
*** two_tired has quit IRC05:00
*** hoonetorg has quit IRC05:00
*** awkwardpenguin has joined #openstack-swift05:11
*** hoonetorg has joined #openstack-swift05:14
*** jeblair has joined #openstack-swift05:14
*** awkwardpenguin has quit IRC05:16
*** jeblair has quit IRC05:20
*** jeblair has joined #openstack-swift05:21
*** jeblair is now known as corvus05:21
*** gyee has quit IRC05:36
*** psachin has joined #openstack-swift05:40
*** pcaruana has joined #openstack-swift06:27
*** pcaruana has quit IRC06:33
*** pcaruana has joined #openstack-swift06:34
*** awkwardpenguin has joined #openstack-swift06:37
*** awkwardpenguin has quit IRC06:42
*** nakagawamsa has joined #openstack-swift06:44
*** awkwardpenguin has joined #openstack-swift07:00
*** awkwardpenguin has quit IRC07:06
*** rcernin has quit IRC07:15
*** m_kazuhiro_ has quit IRC07:24
*** m_kazuhiro has joined #openstack-swift07:24
*** armaan has joined #openstack-swift07:30
*** m_kazuhiro has quit IRC07:35
*** threestrands has quit IRC07:36
*** hoonetorg has quit IRC07:50
*** gkadam has joined #openstack-swift08:04
*** hoonetorg has joined #openstack-swift08:07
*** hseipp has joined #openstack-swift08:12
*** tesseract has joined #openstack-swift08:14
*** geaaru has joined #openstack-swift08:25
*** kei_yama has joined #openstack-swift08:30
*** cshastri has joined #openstack-swift08:34
*** ianychoi has quit IRC08:35
*** hseipp has quit IRC08:47
*** hseipp has joined #openstack-swift08:49
*** kei_yama has quit IRC08:52
*** cbartz has joined #openstack-swift08:56
*** jistr has joined #openstack-swift08:57
*** awkwardpenguin has joined #openstack-swift09:02
*** awkwardpenguin has quit IRC09:07
*** tovin07_ has quit IRC09:32
acolesgood morning09:38
*** threestrands has joined #openstack-swift10:21
openstackgerritChristian Schwede proposed openstack/swift master: Fix exception raise in swift-ring-builder on empty rings  https://review.openstack.org/53248210:25
openstackgerritChristian Schwede proposed openstack/swift master: Change exit code when displaying empty rings  https://review.openstack.org/53248410:36
*** zaitcev_ has joined #openstack-swift10:49
*** ChanServ sets mode: +v zaitcev_10:49
*** zaitcev has quit IRC10:53
*** awkwardpenguin has joined #openstack-swift11:04
*** awkwardpenguin has quit IRC11:09
openstackgerritKazuhiro MIYAHARA proposed openstack/swift master: WIP: Update object expirer to use general task queue system  https://review.openstack.org/51738911:15
*** vint_bra has joined #openstack-swift11:17
kota_good morning acoles11:30
acoleskota_: o/11:30
kota_and then, it's time to leave my office ;)11:30
acoleskota_: goodnight :)11:30
*** armaan has quit IRC11:33
*** armaan has joined #openstack-swift11:33
*** armaan has quit IRC11:34
*** armaan has joined #openstack-swift11:34
*** d0ugal has quit IRC11:39
*** vint_bra has quit IRC11:43
*** m_kazuhiro has joined #openstack-swift11:53
*** m_kazuhiro has quit IRC11:55
*** d0ugal has joined #openstack-swift11:57
*** links has quit IRC12:58
*** hseipp has quit IRC13:01
*** awkwardpenguin has joined #openstack-swift13:06
*** awkwardpenguin has quit IRC13:11
*** mvk has quit IRC13:14
*** awkwardpenguin has joined #openstack-swift13:46
*** awkwardpenguin has quit IRC13:54
*** awkwardpenguin has joined #openstack-swift13:57
*** armaan has quit IRC14:00
*** armaan has joined #openstack-swift14:01
*** mvk has joined #openstack-swift14:22
openstackgerritThiago da Silva proposed openstack/swift feature/s3api: WIP: merge func tests  https://review.openstack.org/53254514:48
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Test that all object methods get shard container update headers  https://review.openstack.org/53254914:55
tdasilvamathiasb: hello, wondering if you have tested swift+barbican integration recently...14:56
-openstackstatus- NOTICE: Gerrit is being restarted due to slowness and to apply kernel patches14:59
*** openstackgerrit has quit IRC15:01
*** ianychoi has joined #openstack-swift15:14
mathiasbtdasilva: hi! I ran some tests with devstack against the pike stable branches in november, but nothing more recent than that15:21
*** aluria has quit IRC15:22
*** aluria has joined #openstack-swift15:22
*** psachin has quit IRC15:22
tdasilvamathiasb: ack, trying to deploy now and i'm running into some errors. I *think* castellan might have broken their api but i could be wrong, investigating...15:24
tdasilvamathiasb: i'm looking at this change: https://github.com/openstack/castellan/commit/17e8b29067e93dbe5f13d0d18e62b06fcd148d3f15:24
tdasilvabut i guess you would have run into this problem with pike...15:25
mathiasbhm, that does somehow ring a bell15:30
mathiasbI'll try to set up my dev env from scratch and see what happens15:31
tdasilvamathiasb: cool, thanks! I tried making a change here https://github.com/openstack/swift/blob/master/swift/common/middleware/crypto/kms_keymaster.py#L6115:32
tdasilvato pass auth_endpoint as part of the context15:32
tdasilvabut now i'm running into some other issue that resembles this https://bugs.launchpad.net/swift/+bug/1662473 :/15:33
openstackLaunchpad bug 1662473 in OpenStack Object Storage (swift) "get internal error when try authenticate by keystone. " [Undecided,Invalid]15:33
tdasilvaso i'm still digging...15:33
notmynamegood morning15:34
notmynamecschwede: I put https://bugs.launchpad.net/swift/+bug/1742417 on the meeting agenda for today15:42
openstackLaunchpad bug 1742417 in OpenStack Object Storage (swift) "swift-ring-builder exit code changed from 0 to 1 for empty rings" [Undecided,New]15:42
*** awkwardpenguin has quit IRC15:52
*** awkwardpenguin has joined #openstack-swift15:54
*** awkwardpenguin has quit IRC15:56
cschwedenotmyname: alright, thx! I'll try to join the meeting then :)15:59
notmynamecschwede: ok. I'll cover that one first, then, since it will be late for you15:59
cschwedenotmyname: :D thanks!15:59
*** awkwardpenguin has joined #openstack-swift16:00
*** itlinux has joined #openstack-swift16:09
* tdasilva is wondering why there's a object-store and object-storage path in the api docs: https://developer.openstack.org/api-ref/16:17
notmynametdasilva: oh weird. looks like maybe something changed about 6 months ago? last time the object-storage was updated16:19
notmynameinteresting. there's also a "shared-file-systems" and "shared-file-system" with the same dates16:20
tdasilvanotmyname: yeah, noticed that too...16:21
*** armaan has quit IRC16:27
*** armaan has joined #openstack-swift16:28
*** armaan has quit IRC16:28
*** armaan has joined #openstack-swift16:28
notmynametdasilva: ok. there's at least an answer :-)16:39
tdasilvanotmyname: yeah, probably a typo in some automation procedure16:40
*** d0ugal has quit IRC16:46
*** awkwardpenguin has quit IRC16:50
*** clarkb has quit IRC16:53
*** gyee has joined #openstack-swift16:54
claygdid cschwede get his ring-builder error code sorted?17:07
*** hseipp has joined #openstack-swift17:09
*** pcaruana has quit IRC17:11
*** saint_ has joined #openstack-swift17:11
*** awkwardpenguin has joined #openstack-swift17:12
zaitcev_clayg: I just put -1 to it, but I may be deeply ignorant about the intent of the current code17:16
zaitcev_I mean the one where you catch that exception17:16
zaitcev_Er, where you catch _other_ exceptions, throw RingEmpty exception, catch that one again17:17
*** JimCheung has joined #openstack-swift17:17
claygzaitcev_: too bad, seemed the regression would indeed be unfortunate/unintentional/problematic... hopefully we can get it sussed out - thanks for looking at!17:17
zaitcev_I think the problem is that things like Tripple-O run the builder thoughtlessly in order to display the current ring status even if it's empty, and then your whole cloud deployment blows up because builder exits with an error.17:18
zaitcev_But that is merely my deduction, I don't know why cschwede needed that fix.17:19
* zaitcev_ puts his Sherlock hat on17:19
*** awkwardpenguin has quit IRC17:22
*** cbartz has quit IRC17:23
*** awkwardpenguin has joined #openstack-swift17:29
*** openstackgerrit has joined #openstack-swift17:34
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Add includes method to ShardRange  https://review.openstack.org/53090117:34
openstackgerritAlistair Coles proposed openstack/swift feature/deep: fixes for sharding/shrinking races  https://review.openstack.org/52993117:34
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Re-introduce shard range updates to root after shard has sharded  https://review.openstack.org/53090217:34
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Add copy method to ShardRange  https://review.openstack.org/53148617:34
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Define entries of shard range db row in one place  https://review.openstack.org/53148717:34
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Add state and state_timestamp to ShardRange  https://review.openstack.org/53148817:34
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Set shard range state during sharding  https://review.openstack.org/53148917:34
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Fix container listing while in sharding state  https://review.openstack.org/53230817:34
*** itlinux has quit IRC17:35
claygwait... did we not merge mattoliverau 's fix for this yet?17:40
*** awkwardpenguin has quit IRC17:40
claygpatch 530839 ZuuuuuUUUUUUUULLLLL!!!17:41
patchbothttps://review.openstack.org/#/c/530839/ - swift - Show devices marked as deleted on empty rings17:41
*** itlinux has joined #openstack-swift17:41
clayg^ cschwede zaitcev_ I think this change gets things going in the right direction17:42
*** awkwardpenguin has joined #openstack-swift17:45
*** gkadam has quit IRC17:47
*** armaan has quit IRC17:47
*** armaan has joined #openstack-swift17:48
*** cshastri has quit IRC17:51
timburkesooo muuuch faaaiiiil17:52
timburkewhat in the world is *this* about? http://logs.openstack.org/84/532484/1/check/openstack-tox-pep8/b4a463c/job-output.txt.gz#_2018-01-10_10_44_22_78719917:52
acolesp 53248417:55
patchbothttps://review.openstack.org/#/c/532484/ - swift - Change exit code when displaying empty rings17:55
timburke> TASK [tox : Run tox without tests]17:56
timburke> pep8: skipped tests17:56
timburke> congratulations :)17:56
timburkei didn't know that smiley could be so... sarcastic17:57
acolestimburke: lol. it should have tongue in cheek17:57
acolestimburke: reminds me of the performance review line 'this employee succeeds at everything they attempt'18:02
*** chsc has joined #openstack-swift18:02
timburkehuh. apparently it's *supposed* to do that? http://logs.openstack.org/47/531447/2/check/swift-tox-py27/4e7a7c7/job-output.txt.gz#_2018-01-09_19_32_08_331823 for example passed just fine18:07
timburkei guess the real trouble is the MODULE FAILURE at http://logs.openstack.org/84/532484/1/check/openstack-tox-pep8/b4a463c/job-output.txt.gz#_2018-01-10_10_44_55_52367918:08
timburkei have *no idea* where ARA found this, but apparently the trouble is "RuntimeError: pip python module is required"18:09
timburkeor, was? looks like at least *some* recently-enqueued jobs are on track to pass18:10
*** geaaru has quit IRC18:15
*** awkwardpenguin has quit IRC18:16
*** awkwardpenguin has joined #openstack-swift18:19
*** openstack has joined #openstack-swift18:31
*** ChanServ sets mode: +o openstack18:31
openstackgerritSamuel Merritt proposed openstack/swift master: Fix object POST of X-Delete-At with skewed clocks  https://review.openstack.org/53259518:35
*** tesseract has quit IRC18:36
*** awkwardpenguin has quit IRC18:55
*** mvk has quit IRC18:57
openstackgerritAlistair Coles proposed openstack/swift feature/deep: WIP Don't attempt to merge root and shard listings in proxy  https://review.openstack.org/53260419:00
*** mvk has joined #openstack-swift19:33
openstackgerritChristian Schwede proposed openstack/swift master: Change exit code when displaying empty rings  https://review.openstack.org/53248419:50
cschwedeclayg, timburke, zaitcev_: thx for the pointers re swift-ring-builder return code/exception fixes - i abandoned one of my proposals and rebased the other one on top of Matts patch19:53
timburkeso much interest in ring-builder usability lately! it's great!19:54
zaitcev_cschwede: I only wanted you to have a look if it's reasonable to return error where we crashed before. Like timburke mentioned that was accidentally undefined. If clayg et al like 0 everywhere, I am fine with that.19:54
zaitcev_enthropy grows where you throw information away by returning the same thing for different situations19:55
zaitcev_kinda19:55
timburkeespecially in light of mattoliverau's change so you can see the devices that were removed, i feel like 0 is fair19:57
acolesmattoliverau: timburke: re ^^ I have been thinking about simplifying listings-while-sharding rather than taking on fixing the merge logic in the proxy, notes here https://etherpad.openstack.org/p/deep-container-listing19:57
acolescc clayg ^19:57
*** awkwardpenguin has joined #openstack-swift20:00
acolesmattoliverau: when I say simplify I mean start with simpler and work towards full feature20:01
acoleswhich I'm hoping will help my small brain grok and the simpler system and make it really robust before we add more complex pieces back in20:02
acolescc torgomatic https://etherpad.openstack.org/p/deep-container-listing20:07
*** awkwardpenguin has quit IRC20:09
*** m_kazuhiro has joined #openstack-swift20:41
m_kazuhirogood morning20:42
*** joeljwright has joined #openstack-swift20:44
*** ChanServ sets mode: +v joeljwright20:44
*** Jeffrey4l has quit IRC20:52
kota_morning20:53
m_kazuhirokota_: mornning20:55
kota_m_kazuhiro: hi20:56
claygtorgomatic: patch 532595 seems less critical than a fix to not create async_pendings to update the expiring_objects queue when the expirer expires an object?!20:59
patchbothttps://review.openstack.org/#/c/532595/ - swift - Fix object POST of X-Delete-At with skewed clocks20:59
claygtorgomatic: unless you already fixed that too20:59
mattoliveraumorning20:59
notmynameswift team meeting time in #openstack-meeting21:00
mattoliverauacoles: I'll give the etherpad another read today21:00
mathiasbtdasilva: with devstack pike, swift master, and castellan==0.7.0 it seems to work21:00
tdasilvamathiasb: one sec21:01
mathiasbtdasilva: but with e.g., the newest castellan I do see some errors (not the same one(s) as you though I think)21:01
tdasilvamathiasb: ah ok21:01
tdasilvai was about to check which castellan I was running21:02
tdasilvamathiasb: curious what errors you are running into21:02
*** Jeffrey4l has joined #openstack-swift21:03
mathiasbERROR castellan KeyManagerError: Key manager error: 'NoneType' object has no attribute 'find'21:04
tdasilvamathiasb: yeah, that was the original error i ran into...21:04
mathiasbah ok, so at least we're probably reproducing the same issue21:05
tdasilvamathiasb: or at least similar one21:05
tdasilvamathiasb: and I tracked that down to castellan trying to deprecate something, but actually breaking it21:05
tdasilvamathiasb: i think it's this change: https://github.com/openstack/castellan/commit/17e8b29067e93dbe5f13d0d18e62b06fcd148d3f21:06
tdasilvaespecifically here: https://github.com/openstack/castellan/commit/17e8b29067e93dbe5f13d0d18e62b06fcd148d3f#diff-fcc75db260ca67948ab5bab26daf7eeeL13721:07
*** zaitcev_ is now known as zaitcev21:12
tdasilvamathiasb: wondering if you could test out a fix I had and see if you run into another problem i'm now running into??21:15
torgomaticclayg: I already fixed that too :) see patch 53238321:16
patchbothttps://review.openstack.org/#/c/532383/ - swift - Don't make async_pendings during object expiration21:16
clayg*phew*21:16
mathiasbtdasilva: of course!21:16
claygok, you're just scraping the bottom of the barrel now - you can't fix ALL the things :P21:16
tdasilvamathiasb: one sec21:17
tdasilvamathiasb: https://paste.fedoraproject.org/paste/sW4eUx4oRnsr9bvYoDYaIA21:19
mathiasbtdasilva: yes, if I also add 'auth_endpoint = http://<keystone_host>/identity' to the [kms_keymaster] section in the proxy conf then it works!21:28
tdasilvamathiasb: without code changes?21:29
mathiasbtdasilva: with you patch21:29
mathiasblet me try without..21:29
tdasilvaah ok21:30
tdasilvamathiasb: can you also tell me what requests and urllib3 package versions you are running?21:30
mathiasbno, without the patch I still get the same error21:30
mathiasbtdasilva: requests==2.18.2, urllib3==1.2221:31
tdasilvamathiasb: ah, interesting...with those versions i'm running into an issue similar to this: https://bugs.launchpad.net/swift/+bug/166247321:34
openstackLaunchpad bug 1662473 in OpenStack Object Storage (swift) "get internal error when try authenticate by keystone. " [Undecided,Invalid]21:34
*** awkwardpenguin has joined #openstack-swift21:34
mathiasbtdasilva: ouch! I am not seeing that21:39
*** awkwardpenguin has quit IRC21:39
*** pcaruana has joined #openstack-swift21:42
*** pcaruana has quit IRC21:43
*** m_kazuhiro has quit IRC21:45
*** dr_gogeta86 has quit IRC21:47
*** alecuyer has quit IRC21:47
*** rledisez has quit IRC21:47
*** armaan has quit IRC21:48
*** alecuyer has joined #openstack-swift21:51
*** rledisez has joined #openstack-swift21:52
*** threestrands_ has joined #openstack-swift21:52
*** threestrands_ has quit IRC21:52
*** threestrands_ has joined #openstack-swift21:52
*** threestrands_ has quit IRC21:53
*** threestrands_ has joined #openstack-swift21:53
*** threestrands_ has quit IRC21:53
*** threestrands_ has joined #openstack-swift21:53
*** threestrands has quit IRC21:54
claygtorgomatic: are you around?  I'm having problems with the probe test on patch 53238321:56
patchbothttps://review.openstack.org/#/c/532383/ - swift - Don't make async_pendings during object expiration21:56
*** joeljwright has quit IRC21:56
notmynameclayg: he had to step out for a bit. may be back online later this afternoon21:57
*** armaan has joined #openstack-swift22:00
*** awkwardpenguin has joined #openstack-swift22:04
zaitcevI do think that CORS is a supremely annoying specification.22:05
openstackgerritTim Burke proposed openstack/swift master: Change exit code when displaying empty rings  https://review.openstack.org/53248422:08
*** rcernin has joined #openstack-swift22:11
timburkeit's messy... and if i tried to do it in python, in func tests that we can actually run in the gate, i'd be forever wondering whether we were actually implementing it right. so, use a Real Browser instead22:12
claygtimburke: sounds exhausting - i would have given up ages ago I think ;)22:13
timburkeafter sufficient bitching and moaning and eventually you start to get pointed in the direction of good ideas :-)22:15
timburkenot *great* ideas (i still don't like that i won't be able to add this to the gate in any sensible way :-/), but tenable ones, and things that are somewhat useful22:16
mattoliverautimburke: step 1, find a way to really test it. Step 2.. n, make it better. You're working on the most important first step :)22:21
*** corvus is now known as jeblair22:21
*** jeblair is now known as corvus22:22
-openstackstatus- NOTICE: The zuul system is being restarted to apply security updates and will be offline for several minutes. It will be restarted and changes re-equeued; changes approved during the downtime will need to be rechecked or re-approved.22:24
*** threestrands_ has quit IRC22:41
*** threestrands has joined #openstack-swift22:55
*** threestrands has quit IRC22:55
*** threestrands has joined #openstack-swift22:55
*** threestrands has quit IRC23:11
*** awkwardpenguin has quit IRC23:18
*** threestrands has joined #openstack-swift23:20
*** kei_yama has joined #openstack-swift23:31
*** chsc has quit IRC23:44
openstackgerritClay Gerrard proposed openstack/swift master: Let recon-cron work with conf.d  https://review.openstack.org/53269623:53

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!