Thursday, 2015-10-01

mattoliverauclayg: o/00:04
mattoliverauacoles_: o/00:04
kota_clayg, acoles: o/00:05
*** chsc has quit IRC00:11
*** lyrrad has quit IRC00:13
*** kota_ has quit IRC00:14
*** zhill has quit IRC00:20
*** garthb has quit IRC00:21
*** dmorita has joined #openstack-swift00:24
*** bill_az has quit IRC00:40
*** m_kazuhiro has joined #openstack-swift00:43
*** nexusz99 has joined #openstack-swift00:55
*** nexusz99 has quit IRC00:55
*** minwoob has quit IRC00:56
*** blmartin_ has quit IRC00:57
*** klrmn1 has quit IRC01:06
*** thumpba has joined #openstack-swift01:14
*** pberis has joined #openstack-swift01:30
*** dustins has joined #openstack-swift01:34
*** thumpba has quit IRC01:38
*** dencaval has quit IRC01:41
*** pberis has quit IRC01:41
*** kota_ has joined #openstack-swift01:42
*** ChanServ sets mode: +v kota_01:42
kota_back again01:42
*** thumpba has joined #openstack-swift01:51
*** jerrygb has joined #openstack-swift02:09
notmynamezigo: just curious, in relation to your email about liberty in debian, what did you use for swift there? we haven't even done an RC for liberty yet, so what does "Liberty RC1 has been fully uploaded..." mean?02:11
*** jrichli has joined #openstack-swift02:18
*** lcurtis has quit IRC02:20
openstackgerritJanie Richling proposed openstack/swift: Decrypting Container Listing  https://review.openstack.org/21443802:32
openstackgerritJanie Richling proposed openstack/swift: Encrypting/Decrypting object metadata  https://review.openstack.org/20331002:32
*** pberis has joined #openstack-swift02:33
hroujrichli, ^ rebase?02:33
jrichlihrou - yes, but i think I have something to fix from that already.  give me a min02:35
hroujrichli, oh great work though !  Yea lots changed from the base patch there02:35
*** sanchitmalhotra has joined #openstack-swift02:44
*** klrmn1 has joined #openstack-swift02:45
*** pberis has quit IRC02:47
*** haomaiwang has joined #openstack-swift02:50
*** haomaiwang has quit IRC02:50
*** haomaiwa_ has joined #openstack-swift02:51
*** wbhuber_ has joined #openstack-swift02:56
*** m_kazuhiro has quit IRC02:59
*** haomaiwa_ has quit IRC03:01
*** haomaiwang has joined #openstack-swift03:01
*** m_kazuhiro has joined #openstack-swift03:03
*** kota_ has quit IRC03:06
*** dustins has quit IRC03:07
*** willsama has joined #openstack-swift03:07
*** silor has joined #openstack-swift03:09
openstackgerritJanie Richling proposed openstack/swift: Encrypting/Decrypting object metadata  https://review.openstack.org/20331003:09
jrichlihrou - thanks.  Let me know if you see issues ^^03:10
*** lcurtis has joined #openstack-swift03:11
hroujrichli, sure thing !  I'll pull those down now.03:11
jrichlihrou: just do that last one right now.  I still gotta rebase the container listing one again ;-)03:11
*** wbhuber_ has quit IRC03:11
hrouhehe sounds good : )  Playing around with the json fix suggested by timburke, so far so good.03:12
jrichligreat!  thanks03:12
*** wbhuber_ has joined #openstack-swift03:12
*** wbhuber_ has quit IRC03:16
*** trifon has joined #openstack-swift03:24
openstackgerritJanie Richling proposed openstack/swift: Decrypting Container Listing  https://review.openstack.org/21443803:25
*** sanchitmalhotra1 has joined #openstack-swift03:31
*** sanchitmalhotra has quit IRC03:33
*** trifon has quit IRC03:33
*** lcurtis has quit IRC03:41
*** sanchitmalhotra has joined #openstack-swift03:51
*** sanchitmalhotra1 has quit IRC03:53
*** links has joined #openstack-swift03:55
*** jerrygb has quit IRC03:57
*** jerrygb has joined #openstack-swift03:58
*** sanchitmalhotra1 has joined #openstack-swift03:59
*** hrou has quit IRC04:00
*** haomaiwang has quit IRC04:01
*** haomaiwang has joined #openstack-swift04:01
*** sanchitmalhotra has quit IRC04:02
*** klrmn1 has quit IRC04:18
*** jrichli has quit IRC04:19
*** willsama has quit IRC04:23
*** mahatic has joined #openstack-swift04:29
*** StevenK has quit IRC04:35
*** StevenK has joined #openstack-swift04:36
*** kota_ has joined #openstack-swift04:36
*** ChanServ sets mode: +v kota_04:36
*** ppai has joined #openstack-swift04:44
*** ktsuyuzaki has joined #openstack-swift04:47
*** kota_ has quit IRC04:49
*** haomaiwang has quit IRC05:01
*** haomaiwang has joined #openstack-swift05:01
*** chsc has joined #openstack-swift05:01
*** trifon has joined #openstack-swift05:09
*** ktsuyuzaki has left #openstack-swift05:22
*** ktsuyuzaki has joined #openstack-swift05:22
*** ktsuyuzaki has left #openstack-swift05:24
*** kota_ has joined #openstack-swift05:25
*** ChanServ sets mode: +v kota_05:25
*** chsc has quit IRC05:26
*** sanchitmalhotra has joined #openstack-swift05:33
*** sanchitmalhotra1 has quit IRC05:35
*** kota_ has quit IRC05:48
*** haomaiwang has quit IRC05:54
*** haomaiwang has joined #openstack-swift05:54
*** trifon has quit IRC06:00
*** haomaiwang has quit IRC06:01
*** haomaiwang has joined #openstack-swift06:01
*** trifon has joined #openstack-swift06:02
*** sanchitmalhotra1 has joined #openstack-swift06:24
*** sanchitmalhotra has quit IRC06:26
*** ktsuyuzaki has joined #openstack-swift06:26
*** ktsuyuzaki has quit IRC06:28
*** sanchitmalhotra has joined #openstack-swift06:32
*** sanchitmalhotra1 has quit IRC06:34
*** ktsuyuzaki has joined #openstack-swift06:36
*** ktsuyuzaki has quit IRC06:36
*** SkyRocknRoll has joined #openstack-swift06:41
*** janonymous has joined #openstack-swift06:51
*** kota_ has joined #openstack-swift06:52
*** kota_ has quit IRC06:53
*** SkyRocknRoll_ has joined #openstack-swift07:00
*** haomaiwang has quit IRC07:01
*** SkyRocknRoll has quit IRC07:01
*** haomaiwang has joined #openstack-swift07:01
*** SkyRocknRoll_ has quit IRC07:01
*** kota_ has joined #openstack-swift07:01
*** SkyRocknRoll_ has joined #openstack-swift07:02
kota_test, test07:02
kota_I've setup znc in my cloud server07:02
*** SkyRocknRoll_ has quit IRC07:04
*** SkyRocknRoll_ has joined #openstack-swift07:04
*** kota_ has quit IRC07:05
*** rledisez has joined #openstack-swift07:05
*** kota_ has joined #openstack-swift07:06
kota_test, again07:06
*** wolsen has quit IRC07:08
m_kazuhirokota_ :test07:08
kota_ok, znc works well in my environment.07:10
kota_I won't lose playbacks anymore, yey. Next, i wann get notifications when someone ping me...07:11
kota_oh yeah, irssi which clayg said seems good.07:15
*** thumpba has quit IRC07:21
*** thumpba has joined #openstack-swift07:25
mattoliveraukota_: nice work! now you can always be pinged :)07:32
mattoliveraukota_: prepare to work more :P07:32
*** eranrom has joined #openstack-swift07:32
*** thumpba has quit IRC07:33
kota_mattoliverau: might be no holidays? :\07:37
kota_kota_: notification test07:40
mattoliveraukota_: notification test 2 :)07:40
kota_mattoliverau: just looking at so no notification at your ping :/07:41
kota_closed my phone, and please note me again.07:41
mattoliveraukota_: ping07:42
kota_mattoliverau: I got it!07:42
mattoliverau\o/07:42
kota_mattoliverau: thanks for the test :)07:42
mattoliverauNps, remember to out your phone on silent ;)07:43
mattoliverauAt night I mean07:43
*** openstackgerrit has quit IRC07:46
*** acoles_ is now known as acoles07:46
*** openstackgerrit has joined #openstack-swift07:46
acoleskota_: good work on bouncer!07:48
*** geaaru has joined #openstack-swift07:51
*** haomaiwang has quit IRC08:01
*** silor has quit IRC08:01
*** haomaiwang has joined #openstack-swift08:01
*** haomaiwang has quit IRC08:03
zigonotmyname: I have a bit given up on Swift package maintenance as someone else who uses it more took over. What he did is just upload of the latest Swift, so that is 2.4.0, and in Sid, not Experimental.08:05
zigoIf you are releasing out of sync, then probably we will also do the packaging out of sync.08:05
*** itlinux has joined #openstack-swift08:06
zigoIt's also not so much of a problem for Swift, because of nicer requirements/dependencies.08:07
*** jordanP has joined #openstack-swift08:08
*** sanchitmalhotra1 has joined #openstack-swift08:18
*** jistr has joined #openstack-swift08:18
*** sanchitmalhotra has quit IRC08:20
*** itlinux has quit IRC08:26
*** thumpba has joined #openstack-swift08:30
*** thumpba has quit IRC08:34
*** sanchitmalhotra has joined #openstack-swift08:36
*** haomaiwang has joined #openstack-swift08:37
*** ho has joined #openstack-swift08:38
*** sanchitmalhotra1 has quit IRC08:39
*** sanchitmalhotra1 has joined #openstack-swift08:56
*** sanchitmalhotra has quit IRC08:58
*** haomaiwang has quit IRC09:01
*** 6A4AALXHF has joined #openstack-swift09:01
openstackgerritAlistair Coles proposed openstack/swift: Fix copy requests to service accounts in Keystone  https://review.openstack.org/22713309:02
*** mac_ified has quit IRC09:02
acolesho: donagh : notmyname ^^ fixes done. thanks for reviews!09:02
*** aix has joined #openstack-swift09:04
hoacoles: thanks for it. looks perfect!09:05
acolesho: great. if you are +1 then perhaps we can ask mattoliverau to take another look09:10
hoacoles: yeah, i'm going to test it now. please wait09:11
acolesho:  sure. thanks.09:11
*** foexle has joined #openstack-swift09:14
openstackgerritAlistair Coles proposed openstack/swift: Add further unit test for cross-prefix copy auth  https://review.openstack.org/22978709:17
acolesho: donagh cschwede mattoliverau notmyname ^^ deliberately a follow-on patch so as to to not delay patch 22713309:17
patchbotacoles: https://review.openstack.org/#/c/227133/ - Fix copy requests to service accounts in Keystone09:17
*** proteusguy_ has quit IRC09:18
*** mahatic_ has joined #openstack-swift09:24
*** mahatic has quit IRC09:24
*** proteusguy_ has joined #openstack-swift09:31
*** marzif has joined #openstack-swift09:32
*** 6A4AALXHF has quit IRC09:52
*** haomaiwang has joined #openstack-swift09:52
acolesho: thanks for testing and +109:52
hoacoles: you are welcome :-)09:53
acolesho: have a good evening09:53
hoacoles: thanks! good morning :-)09:53
acolesdonagh: thanks too for +109:54
acoleshopefully we can land it this week09:54
*** mahatic_ has quit IRC09:58
hodonagh, acoles: we have similar tests for the problem so I would like to arrange them. acoles' unittest is overlapped with donagh's functional test and donagh's functional test is part of my functional test. I think acoles' unittest and my functional test are covered all cases but my patches are big so if we need to commit them ASAP it's better to proceed acoles' unittest and donagh's functional test first.10:00
*** haomaiwang has quit IRC10:01
*** haomaiwang has joined #openstack-swift10:01
acolesho: my unit test is not as good as a func test. last i looked donagh's test needed more work. i wish i had found time to review your func test patches more :(10:03
*** silor has joined #openstack-swift10:06
*** silor has quit IRC10:06
*** sanchitmalhotra has joined #openstack-swift10:12
*** silor has joined #openstack-swift10:12
*** sanchitmalhotra1 has quit IRC10:14
hugokuohad anyone experienced with container-sync SLO object ever ? (Seems not work)10:18
*** silor has quit IRC10:20
*** silor has joined #openstack-swift10:21
onovyacoles, cschwede, clayg, mattoliverau, notmyname: anyone pls? :) https://review.openstack.org/#/c/210736/10:22
*** thumpba has joined #openstack-swift10:32
*** thumpba has quit IRC10:37
*** dmorita has quit IRC10:47
*** acoles is now known as acoles_10:55
hoacoles: ok, now i reviewed your unittest but it's bit difficult to understand the necessity of COPY(L452-455). I think you might want to test internal behavior of copy.10:56
hoacoles: if so, my expectation is response code of the copy is same as response code of last request.10:57
hoacoles: s/my expectation is response/my expectation for response/10:59
*** haomaiwang has quit IRC11:01
*** haomaiwang has joined #openstack-swift11:01
hoacoles: tomorrow I will continue to review it. have a nice day!11:01
*** ho has quit IRC11:02
*** marzif has quit IRC11:04
*** marzif has joined #openstack-swift11:05
vinshWhy am I swifting wrong? --> "ERROR: Invalid Storage Policy Configuration in /etc/swift/swift.conf (Invalid option 'policy_type' in storage-policy section"11:06
vinshIsn't that a valid option?11:07
*** SkyRocknRoll_ has quit IRC11:07
*** acoles_ is now known as acoles11:13
*** marzif has quit IRC11:14
*** mahatic has joined #openstack-swift11:14
*** marzif has joined #openstack-swift11:15
*** marzif has quit IRC11:16
*** marzif has joined #openstack-swift11:16
*** ppai has quit IRC11:17
vinshWhy is policy_type an invalid option?11:17
vinshpolicy_type = replication11:19
acolesvinsh: policy_type only got added as a valid option with erasure codes, so kilo release11:20
vinshah.. 2.2.2-0ubuntu1~cloud011:21
vinshI had just assumed I was already there. d'oh11:21
acoles:)11:21
vinshamateur hour here :)11:21
*** marzif has quit IRC11:26
vinshacoles: Any idea why UCA skipped 2.3.0 ?11:26
*** ccavanna_ has quit IRC11:26
*** marzif has joined #openstack-swift11:26
vinshI see 2.2.2 then 2.4.011:26
*** aix has quit IRC11:27
acolesvinsh: idk sorry11:28
vinshOk thanks. I gather 2.3.0 is kilo.. but don't see it in there.  I'll dig.11:29
acolesyes kilo is 2.3.011:29
vinshMight as well go right to liberty :)11:30
vinshlooks like 2.2.2 is part of the kilo series.. but is pre release.11:30
vinshso no EC.11:30
*** ppai has joined #openstack-swift11:31
*** thumpba has joined #openstack-swift11:33
*** marzif has quit IRC11:36
*** gustavo_ has quit IRC11:37
*** marzif has joined #openstack-swift11:37
*** thumpba has quit IRC11:37
*** kei_yama has quit IRC11:39
vinshCononical seems to have just skipped kilo entirely.  Going to ping them for feedback.11:46
vinshalso, how am I just now noticing this? :)11:46
*** marzif has quit IRC11:46
*** marzif has joined #openstack-swift11:49
vinshacoles: there it is : https://javacruft.wordpress.com/11:52
vinsh"NOTE: We’re shipping Swift 2.2.2 for release – due to the relatively late inclusion of new dependencies to support erasure coding in Swift, we’ve opted not to update to 2.3.0 this cycle in Ubuntu.11:52
vinshLame.11:52
acolesvinsh: interesting11:55
vinshYeah.  Either build your own or be held to their whims I guess.11:56
*** ppai has quit IRC12:00
*** haomaiwang has quit IRC12:01
*** haomaiwang has joined #openstack-swift12:01
*** MVenesio has joined #openstack-swift12:04
*** km has quit IRC12:05
*** marcusvrn_ has joined #openstack-swift12:06
*** marzif has quit IRC12:06
*** marzif has joined #openstack-swift12:07
*** ppai has joined #openstack-swift12:12
*** marzif has quit IRC12:17
*** jerrygb has quit IRC12:17
*** marzif has joined #openstack-swift12:17
*** haomaiwang has quit IRC12:18
*** haomaiwang has joined #openstack-swift12:19
*** alejandrito has joined #openstack-swift12:27
*** thumpba has joined #openstack-swift12:34
*** m_kazuhiro has quit IRC12:44
*** mahatic has quit IRC12:46
*** thumpba has quit IRC12:50
*** bill_az has joined #openstack-swift12:53
*** delattec has joined #openstack-swift12:56
*** cdelatte has quit IRC12:58
*** mac_ified has joined #openstack-swift12:58
*** mahatic has joined #openstack-swift12:59
*** haomaiwang has quit IRC13:01
*** haomaiwang has joined #openstack-swift13:01
*** pberis has joined #openstack-swift13:04
*** eranrom has quit IRC13:08
*** hrou has joined #openstack-swift13:11
*** ppai has quit IRC13:13
*** dustins has joined #openstack-swift13:18
*** marzif has quit IRC13:20
*** wbhuber_ has joined #openstack-swift13:20
*** marzif has joined #openstack-swift13:20
*** marzif has quit IRC13:25
*** DjamOlsky has joined #openstack-swift13:25
*** trifon has quit IRC13:28
*** jrichli has joined #openstack-swift13:33
*** wbhuber_ has quit IRC13:39
*** wbhuber_ has joined #openstack-swift13:40
*** wbhuber_ has quit IRC13:44
vinshIs the ec_object_segment_size buffered in proxy memory?13:47
vinshI set that to 1G.. vs default.. but don't see any more memory consumed.13:48
*** ccavanna has joined #openstack-swift13:48
vinsh(after restarting services of course)13:48
wbhuberclayg: peluse: acoles: when you have a cluster running with replication in production, how do we migrate from a replication-only policy to multiple policies running on cluster without stopping any services?13:50
wbhuberI assume stop one or two of the proxy servers (at least three) to ensure that transacations are kept running for new/lookup objects13:50
vinshCould just do a graceful reload.13:51
wbhubervinsh: what are the steps to do a graceful reload?13:51
vinshswift-init reload vs vestart13:51
vinsh*restart13:51
vinshTake a look at the help output for swift-init13:51
wbhubervinsh: did you try testing that yet?13:52
vinshhow so?13:52
vinshYour going to have to restart services to load the new config.. thats a smoother way13:52
wbhuberbut that'd cease some of the operations that are running in the meantime, correct?13:53
vinsh"          reload: graceful shutdown then restart on supporting servers"13:53
wbhuberthat's in production - we would like to have no downtime during migration13:53
vinshmeaning.. it lets current requests complete before reloading13:53
vinshone proxy reloads the new config.. then the other. no down time.13:54
wbhuberok, that sounds feasible13:54
vinshare you using puppet or?13:54
vinshwhat are you configuring swift with?13:54
wbhuberansible if i understood your Q right13:55
vinshOk, that should work well with ansible.  push config out. reload proxy one. then reload proxy two.13:55
vinshTake a peek at the docs for swift-init... its got all the magic you need.13:56
wbhubervinsh: thumbs up13:56
*** links has quit IRC13:57
*** aix has joined #openstack-swift14:00
*** haomaiwang has quit IRC14:01
*** haomaiwang has joined #openstack-swift14:01
*** willsama has joined #openstack-swift14:07
*** lyrrad has joined #openstack-swift14:09
*** david-lyle has quit IRC14:11
*** david-lyle has joined #openstack-swift14:13
*** mahatic has quit IRC14:14
*** mahatic has joined #openstack-swift14:14
openstackgerritMahati Chamarthy proposed openstack/swift: Guess content type for PUT and probe tests  https://review.openstack.org/22089714:24
openstackgerritMahati Chamarthy proposed openstack/swift: Modified tests to include real crypto  https://review.openstack.org/21145114:24
openstackgerritMahati Chamarthy proposed openstack/swift: Encrypt flag for probe tests  https://review.openstack.org/21697414:24
*** thumpba has joined #openstack-swift14:26
*** thumpba has quit IRC14:26
*** thumpba has joined #openstack-swift14:26
vinshI see my memory being consumed up to segment size now.. all is well.14:34
*** jlhinson has joined #openstack-swift14:39
*** thurloat is now known as thurloat_isgone14:42
wbhubervinsh - an object up to 1 gb consuming your memory accordingly?14:42
vinshI think14:43
vinshyeah. failed.. but thats probably due to max chunk size in general.14:43
wbhuber64k chunk size?14:43
vinshI think 5G14:45
wbhuberwhat happens if you lower the chunk size but leave the seg size as is14:45
vinshNot sure, have to work on other stuff first :)14:45
wbhuber:)14:45
notmynamegood morning14:53
vinsh'mornin14:54
*** minwoob has joined #openstack-swift14:58
*** haomaiwang has quit IRC14:59
*** haomaiwang has joined #openstack-swift15:00
*** sayali has quit IRC15:00
*** haomaiwang has quit IRC15:01
*** haomaiwa_ has joined #openstack-swift15:01
notmynameso canonical hasn't released anything for swift more recent than 2.2.2 because they don't think we're a responsible set of devs who know how to have a beta feature?15:02
*** sayali has joined #openstack-swift15:02
*** haomaiwa_ has quit IRC15:04
*** wolsen has joined #openstack-swift15:05
*** nadeem has joined #openstack-swift15:08
*** nadeem has quit IRC15:11
vinshI guess... i'm grumpy at canonical for that.15:12
vinshThey have a 2.4.0 out there now.15:12
vinshBut skipped 2.3.0 entirely15:12
notmynamevinsh: I'm not surprised that they may skip versions15:13
vinshyeah, and thats the cost I pay for consuming their packages vs building my own.15:13
notmynameswift has historically released between the openstack integrated releases, and also at the integrated release15:13
vinshSaves time in general15:13
notmynamedistros afaik just take the integrated one15:13
notmynamehowever, now in openstack there isn't an "integrated release". and many of the other projects are also doing mid-cycle releases15:14
notmynamewhat I'm annoyed at is that we spend extra effort to do the end-of-cycle release and distros seemingly skip it or don't pick it up and otherwise say "swift is different so deal with it" (which I reject)15:15
vinshI'm with ya on that. :-/15:16
vinshI would have been more vocal with them had I realized it was the case back when.15:16
vinshFor now, I'll run 2.4.0 on my EC cluster.  Move 2.4.0 without EC to production after a little soak time.15:18
notmynamecool15:18
notmynamemany people are using 2.4.0. nobody is using ec in prod. that's ok15:18
vinshthis EC cluster is a POC to eval some beefy nodes we bought.15:18
*** jerrygb has joined #openstack-swift15:19
notmynameI think we'll start seeing people use ec in prod after this next release15:19
vinshif this hardware pans out.. we will be buying about 10x more of it for a larger EC poc15:19
*** jerrygb has quit IRC15:19
notmyname(after the current priority patches land)15:19
notmynamevery cool15:19
vinshI'm excited :)15:19
*** jerrygb has joined #openstack-swift15:19
*** MVenesio has quit IRC15:22
*** MVenesio has joined #openstack-swift15:23
*** chsc has joined #openstack-swift15:25
*** zhill has joined #openstack-swift15:31
*** chsc has quit IRC15:32
*** DjamOlsky has quit IRC15:33
openstackgerritAlistair Coles proposed openstack/swift: Enable object server to return non-durable data  https://review.openstack.org/21527615:47
openstackgerritAlistair Coles proposed openstack/swift: Add POST capability to ssync for .meta files  https://review.openstack.org/13849815:47
*** nadeem has joined #openstack-swift15:48
*** rohit_ has joined #openstack-swift15:48
jrichlimahatic : I saw that you mentioned patch 220897 resolves the TestSerialization issue.  Sorry I haven't reviewed that yet.  I will do that very soon :-)15:52
patchbotjrichli: https://review.openstack.org/#/c/220897/ - Guess content type for PUT and probe tests15:52
*** Fin1te has joined #openstack-swift15:56
*** chsc has joined #openstack-swift15:58
*** lyrrad has quit IRC15:58
*** MVenesio has quit IRC15:58
*** alejandrito has quit IRC15:58
*** chsc has quit IRC16:03
*** MVenesio has joined #openstack-swift16:05
*** alejandrito has joined #openstack-swift16:05
*** nadeem has quit IRC16:07
*** nadeem has joined #openstack-swift16:08
*** breitz has quit IRC16:14
*** breitz has joined #openstack-swift16:14
*** rledisez has quit IRC16:24
*** blmartin has joined #openstack-swift16:27
*** garthb has joined #openstack-swift16:28
*** jistr has quit IRC16:29
*** silor has quit IRC16:29
*** trifon has joined #openstack-swift16:34
*** lyrrad has joined #openstack-swift16:34
*** lyrrad has quit IRC16:35
*** lyrrad has joined #openstack-swift16:43
*** amit213 has joined #openstack-swift16:43
*** garthb has quit IRC16:46
*** mahatic has quit IRC16:47
*** chsc has joined #openstack-swift16:47
notmynameacoles: I /think/ I'm understandign what's going on with the keystone patch. but your subsequent test leaves me with a quesion https://review.openstack.org/#/c/229787/16:56
acolesnotmyname: yeah i just saw your review16:56
*** Fin1te has quit IRC16:57
acolesthere's no real proxy in the tests so the headers are irrelevant tbh16:57
notmynamemostly I'm trying to figure it out rather than say it's good or bad16:57
acolesits not a great test16:57
*** chsc has quit IRC16:58
notmynameok16:58
notmynameso why the Destination headers to start with?16:58
acolesi said it simulates a COPY - basically it says 'if i use a request env that get's allowed, then re-use the SAME env with a different, disallowed path, it should get denied16:59
acoleswhy Destination header? ummm, to make it look authentic i guess :) on reflection maybe it just confuses - ho also tripped up on that patch17:00
notmynamehe17:00
notmyname*heh17:00
acolesa func test would be MUCH better17:00
notmynameyeah17:00
acolesso I am just reviewing patch 202411 again cos that is the start of v comprehensive access control testing suite17:01
patchbotacoles: https://review.openstack.org/#/c/202411/ - Add functional test for access control (PUT/DELETE...17:01
acolesnotmyname: unfortunately we need to do some devstack work to get these tests (and other existing tests) to run on jenkins17:02
notmynameah17:02
acolesor we get charz to set it all up on community cluster :P17:02
notmyname:-)17:02
notmynameI'd prefer devstack to start with, depending on what that takes17:03
*** chsc has joined #openstack-swift17:03
acolesagree. I would do it once "everything else" is done. I think maybe ho would take it on.17:04
*** jerrygb has quit IRC17:04
notmynamedespite the lack of functional tests, patch 227133 seems ok. the bug makes sense. the patch seems right. just the whole service vs operator roles and who sets what when seems to make my brain stutter17:04
patchbotnotmyname: https://review.openstack.org/#/c/227133/ - Fix copy requests to service accounts in Keystone17:04
acolesnotmyname: consider it a gentle warm up for 21527617:05
notmynamelol17:05
notmynameyeah, I should move on now17:05
charzacoles: need add something or?17:05
notmynamecharz: nah, not yet :-)17:05
charznotmyname: k17:06
notmynamecharz: was there something going on with the QA cluster recently? did you fix something?17:06
notmynameseemed like it hadn't been reporting, but now it is17:06
acolescharz: no, we were just kidding. thanks.17:06
charznotmyname: idk, the VM just no response. I can't reach the VM, so I restart it.17:07
notmynameacoles: I think I need to take the 2 patches as one. look at just patch 215276 and make sure both together make sense17:07
patchbotnotmyname: https://review.openstack.org/#/c/215276/ - Enable object server to return non-durable data17:07
notmynamecharz: ah, ok. thanks17:07
notmynameacoles: does that make sense from your perspective? will I miss anything by doing that?17:09
charzacoles: ha, it's good to see my nickname in irc again. :-P17:09
acolesnotmyname: makes sense. there are three main areas of change: obj.py - the get response handling 'brains', ssync - do POSTs, then diskfile.py for the enabling changes: make richer info about on disk files available, open diskfile without a durable etc.17:11
acolesnotmyname: to start with, ignore the ssync stuff, concentrate on the proxy <-> obj server GET interaction17:12
notmynameok17:12
notmynameyeah, from looking at it earlier (from a high level), the concept makes a lot of sense. I'm just worried about missing nuances and systemic effects17:13
acolesme too :)17:13
notmynamebut that's why we don't stop writing code, I guess. we find a bug and fix it17:13
*** SkyRocknRoll has joined #openstack-swift17:14
acolesnotmyname: once you get to wondering why the ssync change is needed, run nosetests test.probe.test_reconstructor_rebuild:TestReconstructorRebuild.test_main on master and take a look at your filesystem for .data files at different timestamps on different nodes :/17:14
notmynameright. it's stuff like that that I'm worried about :-)17:15
notmynameI totally get how that's a thing. but would I have caught that? I'm not sure17:15
acolesnotmyname: arguably even if 215276 doesn't make it we should really try to land patch 138498 - it scares me that real clusters will start to accumulate diverse timestamps for objects and then it'll be hard to put a fix into place when there are already confused timistamps.17:17
patchbotacoles: https://review.openstack.org/#/c/138498/ - Add POST capability to ssync for .meta files17:17
acolesnotmyname: i'll be away for a couple of hours, school thing to attend, but i'll check back later in case anyone has questions or wants to flame me ;)17:18
notmyname:-)17:19
*** SkyRocknRoll has quit IRC17:28
*** diogogmt has joined #openstack-swift17:32
diogogmtnotmyname: thanks for the links, initially i was trying to use the swift python client in debug mode to find the sequence of requests executed when uploading a file, however, i couldn’t find much info, this is what i got from the debug logs: http://pastebin.com/exBmNDbu17:35
diogogmtthe actual file was uplodaded by i don’t see any indication of the request that uploaded the file in the debug logs17:35
*** alejandrito has quit IRC17:36
notmynamediogogmt: the equivalent curl commands are printed in that debug output17:36
notmynamediogogmt: eg line 2917:37
*** klrmn1 has joined #openstack-swift17:37
*** alejandrito has joined #openstack-swift17:38
* notmyname steps away for a bit17:38
*** alejandrito has quit IRC17:46
*** alejandrito has joined #openstack-swift17:47
*** garthb has joined #openstack-swift17:47
*** NM has joined #openstack-swift17:51
*** jordanP has quit IRC17:52
*** lcurtis has joined #openstack-swift17:54
diogogmtnotmyname: but where is the file content transfered in the request? is it set in the headers or in the body?17:56
*** Fin1te has joined #openstack-swift17:58
*** cdelatte has joined #openstack-swift18:06
*** delattec has quit IRC18:08
*** geaaru has quit IRC18:21
*** delattec has joined #openstack-swift18:22
*** cdelatte has quit IRC18:24
*** aix has quit IRC18:26
*** marzif has joined #openstack-swift18:27
*** cdelatte has joined #openstack-swift18:30
*** openstackgerrit has quit IRC18:31
*** openstackgerrit has joined #openstack-swift18:31
*** delattec has quit IRC18:32
openstackgerritClay Gerrard proposed openstack/swift: Add search filter examples to swift-ring-builder dispersion help  https://review.openstack.org/22705818:34
*** sanchitmalhotra1 has joined #openstack-swift18:38
*** sanchitmalhotra has quit IRC18:40
*** CR7 has joined #openstack-swift18:43
*** marzif has quit IRC18:45
*** delattec has joined #openstack-swift18:51
*** delatte has joined #openstack-swift18:51
*** eranrom has joined #openstack-swift18:52
*** cdelatte has quit IRC18:53
*** chao_li has joined #openstack-swift18:54
*** cdelatte has joined #openstack-swift18:55
chao_liHi, nice to meet you guys, just wonder have any of you guys integrated CADF with swift before18:55
*** delattec has quit IRC18:56
*** delatte has quit IRC18:56
notmynamediogogmt: the file content is transferred in the request body18:56
notmynamediogogmt: so eg to upload something with curl, use the -T flag and give it a local file18:57
diogogmtnotmyname: so are the debugs logs complete? shouldn’t curl have more arguments?18:57
diogogmtdo i need to set the transfer encoding to header to chunked as well as the encoding type to gzip?18:58
notmynamediogogmt: they are there for a guide, definitely. it's not actually using curl under the covers, and they are there as a hint or guideline18:58
*** thumpba has quit IRC18:58
*** thumpba_ has joined #openstack-swift18:58
*** delattec has joined #openstack-swift18:59
diogogmtnotmyname: ok, that makes sense — so the api exects to receive a chunked multi part PUT/POST request?19:00
*** lcurtis has quit IRC19:00
*** rohit_ has quit IRC19:00
*** cdelatte has quit IRC19:01
*** pberis has quit IRC19:01
*** pberis has joined #openstack-swift19:02
notmynamediogogmt: swift supports both content-length and chunked transfer encoding19:03
notmynamediogogmt: eg here's somethign I had saved a few days ago for something I was working on. (I don't even remember what) https://gist.github.com/notmyname/11ed48cdb554085e830719:04
diogogmtnotmyname: thanks, i’m trying to gather some info about how the swift REST api works to later on connect to a node-js application19:05
notmynamecool19:05
notmynamethose links I gave you earlier are the best place to start. they show requests and responses19:05
notmynamediogogmt: http://www.amazon.com/OpenStack-Swift-Administering-Developing-Storage/dp/1491900822/ref=sr_1_119:08
notmynamediogogmt: that has several chapters on the swift api19:08
diogogmtnotmyname: great! that’s really handy, thanks19:08
notmynamediogogmt: also, you could request it from https://swiftstack.com/books/ (disclosure: where I work)19:08
diogogmtnotmyname: do you have any promo codes i could use there?19:09
notmynamediogogmt: if you request it from swiftstack, we'll mail you one19:09
*** bill_az has quit IRC19:17
*** david-lyle has quit IRC19:18
*** ccavanna has quit IRC19:25
*** david-lyle has joined #openstack-swift19:27
openstackgerritTim Burke proposed openstack/python-swiftclient: Make LengthWrappers resettable if their _readable is  https://review.openstack.org/22689719:28
*** delatte has joined #openstack-swift19:30
*** cdelatte has joined #openstack-swift19:30
openstackgerritTim Burke proposed openstack/python-swiftclient: Add py35 to default tox environments  https://review.openstack.org/23010019:32
*** delattec has quit IRC19:33
*** tellesnobrega has quit IRC19:33
*** thumpba_ has quit IRC19:37
*** itlinux has joined #openstack-swift19:44
*** logan2 has quit IRC19:47
*** bill_az has joined #openstack-swift19:48
*** [1]eranrom has joined #openstack-swift19:52
*** eranrom has quit IRC19:52
*** [1]eranrom is now known as eranrom19:52
*** logan2 has joined #openstack-swift19:55
*** lcurtis has joined #openstack-swift19:56
claygpy35!?  what are we crazy!?20:01
notmynamethat's like 32 versions from now20:02
notmynamehttp://d.not.mn/tshirt_hint.jpg20:03
jrichliwhat a tease!20:04
claygnotmyname: it's not loading for me :'(20:04
notmynameclayg: took a while for me too20:04
notmynamemlanner: did you break the office proxy? ;-)20:04
jrichliyou've got me wondering where the bird factors in20:05
hrouOr the Japanese word for bird (but maybe that was just me assuming : - )20:06
jrichliwell, i guess those silver lines could be the bottom of a wing20:06
jrichlinah ..20:06
jrichlimaybe its white20:07
*** thumpba has joined #openstack-swift20:08
*** thumpba has quit IRC20:08
*** thumpba has joined #openstack-swift20:09
*** Fin1te has quit IRC20:10
*** thumpba has quit IRC20:11
mlannernotmyname: nope. wfm. i was going to remove the caching, but haven't ... yet.20:14
*** thumpba has joined #openstack-swift20:19
*** thumpba has quit IRC20:20
claygI wonder if there's a backwards compatible way for http clients & servers to negotiate a request pipeling for requests that return 4/5XX instead of 100-continue that wouldn't require the body...20:20
clayglike instead of Expect: 100-continue it's like Require: 100-continue - and if if you give me anything else I'm just going to drop the the next request on the wire - and not send you any body.20:21
acolesnotmyname: is that part of a 'do not tumble dry' symbol? ;-)20:22
claygI think maybe the symantic might also require that instead of timeing out waiting on the 100 line and sending the body anyway - you have to close the connection and retry with expect 100 continue.20:22
clayglike - either tell me I can proceed or I'm not sending it - and if you say don't I won't - promise20:23
claygRequire: 100-continue - should totally be a thing20:23
clayg(everytime I tcpdump ssync I have these thoughts)20:23
claygwe need to get mark nottingham on this20:23
claygor write Sweventlet so we can add SPDY support - or make up our own thing with json (CC/troll redbo)20:24
jrichliacoles: lol.20:24
acolesjrichli: heh, well you know notmyname is playing with us20:26
notmyname:-)20:28
acolesclayg: ok i'm curious - ssync & Expect 100 ?? is that a thing?20:29
*** thumpba has joined #openstack-swift20:29
acolesclayg: thansk for that review btw20:29
*** CR7 has quit IRC20:30
claygacoles: 'a161102fba1710ef912af194b8d4635a dm' - rly?  like you're totally schizophrenic in your "invent my own encoding" vs. "just use json"20:30
*** CR7 has joined #openstack-swift20:30
claygI can't wait to go read this packing and unpacking parser20:31
claygacoles: NO!20:31
claygacoles: RE: ssync & expect 10020:31
acolesclayg: omg i was just waiting for that :) had same thought myself when i looked over it today20:31
claygacoles: but like the hummingbird replicator does it way better20:31
acolesclayg: phew, thought i had missed something there20:31
claygacoles: instead of all the back and forth "within" the SSYNC request it just does pipelined SYNC requests (which are basically PUT's to a timestamp"20:32
clayger... PUT to a hash20:32
acolesclayg: right. makes sense.20:32
claygbut it's cool about 409's and shit20:32
acolesyup20:32
*** thumpba has quit IRC20:32
claygbut it doesn't work with HTTP because you can't do pipelined requests like that :'(20:32
claygso it opens the socket and does the REPLICATE, then it just start's sending this '<length>\r\n<json-version-of-http-headers>\r\n[WAIT FOR JSON OK]<body>\r\n<next-request-length>\r\n><json-headers>\r\n[WAIT FOR JSON 409]\r\n...'20:34
claygunless redbo re-wrote it again20:34
clayganyway - totally better than what we've got with SSYNC20:34
acoleshrm. reminds me, i wondered if we should make the ssync_sender tolerate response content between the updates_start and updates_end so that one day we can start to send back subrequest success/fail info without raising exceptions in legacy senders.20:35
claygacoles: maybe - I think we should just re-work the protocol tho :\20:35
claygacoles: or go ahead and build something compatible with the hummingbird replciator (wouldn't be so bad neway)20:36
acolesso they send *json* ;)20:36
claygheh20:36
clayga) why are you still working b) what do you call the ssync missing check line encoding?20:36
acolesi think i just use encode/decode for anything that transforms stuff to a string or w/e. i should get a thesaurus.20:37
clayga161102fba1710ef912af194b8d4635a 1443730105.90698 m:18a915 and the response a161102fba1710ef912af194b8d4635a dm (vs. just a161102fba1710ef912af194b8d4635a m)20:38
acoles?20:39
acolesre a) i came back to switch off my laptop!20:39
clayglol20:39
*** breitz has quit IRC20:41
acolesseriously, the more i think about it the more i feel we should really try to land 138498 even if 215276 is a stretch, cos if we have real clusters spreading mixed timestamps for frags around then 215276 will never work on those clusters.20:42
claygacoles: yeah it's all good - we'll totally get it in20:42
clayghash is a builtin name btw - so you've got at least one push-over coming20:43
acolesclayg: hey thanks for looking at it and, you know, just tell me what can be better, i'm all ears20:43
acolesheh. ok there's one already20:43
claygacoles: i'll just push over - don't worry about it20:43
acolesyeah, i used object recently, and dir. :/ fixed those though20:44
claygI'ma wait till I make it through tho20:44
clayglol20:44
acolessure20:44
claygacoles: get some better syntax highlighting bro20:44
acolessurprised my IDE didn't yell at me20:44
claygacoles: what editor do you use?20:44
acolesor maybe my old eyes just don't see the colors no more20:44
acolespycharm20:44
claygrofl20:44
acolesits all shades of grey20:45
acolesno, wait, bad choice20:45
claygjeez20:45
openstackgerritSamuel Merritt proposed openstack/swift: Improve SLO PUT error checking  https://review.openstack.org/22809920:46
*** bill_az_ has joined #openstack-swift20:46
*** thumpba has joined #openstack-swift20:47
claygacoles: so how come old ssync_reciver's don't blow up when the line has the extra "metadata delta" bit?20:47
acolesclayg: ok i need some sleep, forgive my blunders, change it all, particularly header names. re-author the commit to yourself and I'll +2A.20:47
* acoles spreads the blame20:47
claygacoles: RIGHT ON!20:47
claygnone of us are as dumb as all of us!20:47
notmyname:-)20:47
acolesclayg: oh, remember crazy time when we landed EC, well i sneaked in a change to ssync_receiver to tolerate extra parts in the missing check phrases.20:48
acolesclayg: that was the best thing i did on EC!20:48
acolesi saw this coming20:48
claygacoles: yeah I thought it was something like that - nice work20:48
acolesand i think i try to do same here, like look for what i expect but don't blow up if there is other stuff20:49
claygobject_hash, timestamp = [urllib.unquote(v) for v in parts[:2]]20:49
clayg^ boom!20:49
acoleswell some pieces can't change ;)20:50
*** delatte has quit IRC20:50
*** cdelatte has quit IRC20:50
claygno no, that's the bit on master, it's great20:50
clayggo to bed or something - stop listening to me20:50
acolesk. got it. go away coles20:51
acolesclayg: ok. have a good one.20:52
*** [1]eranrom has joined #openstack-swift20:52
*** eranrom has quit IRC20:52
*** [1]eranrom is now known as eranrom20:52
acoleshuh, i remembered why i came here:20:52
acolesmattoliverau: if you have chance could you take another look at https://review.openstack.org/#/c/227133/ it has one +2. thanks. hope your phone is on silent :D20:53
*** acoles is now known as acoles_20:54
*** ekarlso has quit IRC20:55
openstackgerritSamuel Merritt proposed openstack/swift: Improve SLO PUT error checking  https://review.openstack.org/22809920:56
*** ekarlso has joined #openstack-swift20:58
*** itlinux has quit IRC21:01
*** itlinux has joined #openstack-swift21:01
*** bill_az has quit IRC21:08
*** Fin1te has joined #openstack-swift21:13
*** chao_li_ has joined #openstack-swift21:13
*** dustins has quit IRC21:15
*** pberis has quit IRC21:18
openstackgerritTim Burke proposed openstack/swift: Fix slorange on-disk format when including whole object  https://review.openstack.org/23015221:19
timburketorgomatic: ^^^21:19
torgomatictimburke: you should really put some words in there about how the current behavior is only wasteful, not harmful21:20
torgomaticit sounds way scarier than it is :|21:20
timburkegood point21:20
openstackgerritTim Burke proposed openstack/swift: Fix slorange on-disk format when including whole object  https://review.openstack.org/23015221:24
*** wbhuber has quit IRC21:25
notmynameclayg: you're working on a push for patch 138498? if so, I found a typo that needs fixed21:25
patchbotnotmyname: https://review.openstack.org/#/c/138498/ - Add POST capability to ssync for .meta files21:25
*** trifon has quit IRC21:26
*** zhill has quit IRC21:28
*** chao_li_ has quit IRC21:30
*** chao_li has quit IRC21:30
claygnotmyname: yup - just stick it in a comment and I'll add it21:34
claygor push over and I'll rebase before I push fixes21:35
notmynamewill do. still going through it. simple thing, so if you were about to push something I wanted to make sure you got it21:35
*** jrichli has quit IRC21:35
notmynameclayg: in test_diskfile.py, in test_get_metafile_not_opened(), the parens are wrong21:36
claygnotmyname: can that be true?  sounds like a bs test if it's wrong and not failing...21:41
*** MVenesio has quit IRC21:41
notmynameI'm surprised "self.assertRaises(Exception)" doesn't fail since it only has the one argument. but assuming it doesn't, then the line just ends up being a tuple21:42
notmynamewhich isn't wrong, syntactically21:42
notmynamesomethingsomething type systems21:42
*** zhill has joined #openstack-swift21:43
notmynameof course there are 2 forms of assertRaises. why wouldn't there be? https://docs.python.org/2/library/unittest.html#unittest.TestCase.assertRaises21:43
*** david-lyle has quit IRC21:43
*** eranrom has quit IRC21:49
*** ccavanna has joined #openstack-swift21:51
*** Fin1te has quit IRC21:52
claygwell *yeah*21:54
claygnotmyname: so is the test wrong or not?21:54
notmynameother than the parens?21:54
notmynameI think it's fine, just needs the right parens to actually check something21:54
claygnotmyname: you might need to send me a diff - did you make a comment on the patch?21:54
notmynameself.assertRaises(DiskFileNotOpen), df.get_metafile_metadata   # current line21:55
notmynameself.assertRaises(DiskFileNotOpen, df.get_metafile_metadata)   # new line21:55
claygnotmyname: this is patch 13849821:55
patchbotclayg: https://review.openstack.org/#/c/138498/ - Add POST capability to ssync for .meta files21:55
notmynameyup21:55
claygtest.unit.obj.test_diskfile you say?21:56
notmynameyup21:56
claygthis is in the diff - or preexisting?21:56
notmynametest_get_metafile_metadata_not_opened21:56
notmynameit's a test added in this patch21:56
*** willsama has quit IRC21:56
*** david-lyle has joined #openstack-swift21:56
claygOMG21:57
claygfuck this new gerrit view21:57
claygctrl-F doesn't work21:57
notmynamelol, you haven't turned it off yet?21:57
claygwell I'm going to NOW21:57
claygwhat fucking stpuid - why - I don't even21:58
*** foexle has quit IRC22:01
claygi just can't - i try to read through gerrit's issues but there are just so many of them - i ... i'm full of hate22:02
*** CaioBrentano has quit IRC22:06
*** CaioBrentano has joined #openstack-swift22:06
*** CaioBrentano has quit IRC22:07
*** CaioBrentano has joined #openstack-swift22:08
*** bill_az has joined #openstack-swift22:10
*** jlhinson has quit IRC22:11
claygtorgomatic: you remember that one time we had that bug environ caches and we were going to fix everything but then didn't because simplified diffs - but then never went back and made any of the other good changes?22:16
clayg... cause that was great22:16
torgomaticyeah, that would be nice to clean up at some point22:16
claygtorgomatic: do we have a lp bug or *anything*22:17
torgomaticeh, I didn't make one22:17
claygsweet - me either!22:17
claygnotmyname: do you remember the container-tempurls security patch with like 86 comments?22:17
notmynameyeah22:17
claygnotmyname: in some of those comments was some great code that we opt'd not to merge with that fix - but we should totally caputre somewhere to go back and add all that22:18
*** NM has quit IRC22:18
claygjust thought of that because I'm looking at another change with risk of subtle bugs because of leaking identity on dicts :'(22:18
notmynamehttps://bugs.launchpad.net/swift/+bug/144921222:19
openstackLaunchpad bug 1449212 in OpenStack Object Storage (swift) kilo "Container level temp URLs can unintentionally leak data." [Undecided,Fix committed]22:19
*** lcurtis has quit IRC22:25
claygtorgomatic: notmyname: lp bug #150193922:25
openstackLaunchpad bug 1501939 in OpenStack Object Storage (swift) "get_info environ caching is leaky and brittle" [Undecided,New] https://launchpad.net/bugs/150193922:25
notmynameclayg: thanks22:26
*** thumpba has quit IRC22:38
*** thumpba has joined #openstack-swift22:41
*** thumpba has quit IRC22:42
*** MVenesio has joined #openstack-swift22:49
*** chsc has quit IRC22:49
*** MVenesio has quit IRC22:53
*** zhill has quit IRC22:54
*** hrou has quit IRC22:55
*** alejandrito has quit IRC23:03
*** chlong has quit IRC23:07
*** km has joined #openstack-swift23:08
*** itlinux has quit IRC23:13
*** CR7 has quit IRC23:19
*** blmartin has quit IRC23:20
*** fbo has quit IRC23:22
*** kei_yama has joined #openstack-swift23:30
*** CR7 has joined #openstack-swift23:36
*** gyee has joined #openstack-swift23:39
*** minwoob has quit IRC23:48
*** wbhuber has joined #openstack-swift23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!