Wednesday, 2015-12-16

mattoliveraugmmaha: pudb not working is fine00:00
gmmahamattoliverau: :D it helps dumb ppl like me a lot00:00
*** lcurtis has quit IRC00:01
mattoliveraulol, just use, pudb works if you use the 'python -m testtools.run discover --load-list my-list' technique, and that works good enough :(00:01
mattoliveraus/:(/:)00:01
hogood morning!00:02
notmynameboth normal and in-process runs completed on my saio (in-process was faster). both successful00:02
notmynamethat's a first00:02
mattoliverauvinsh: The subrequests *LO users to grab the segments don't include the referrer in the request, so adding the *.example.com referrer to the segments container doesn't work. Making it .r:* is working cause it accepts everything.00:04
*** dmorita has quit IRC00:04
mattoliveraunotmyname: I know, it scares me too. Looks like logging gives a performance hit00:05
*** dmorita has joined #openstack-swift00:05
*** nadeem has quit IRC00:06
gmmahamattoliverau: ohh didnt know that.. thanks :D00:06
notmynamegmmaha: where is SWIFT_TEST_DEBUG_LOGS set?00:07
notmynamegmmaha: ah, I see. defaults to off (undefined == off)00:08
notmynameonly set it to turn it on00:08
notmynamenm00:08
gmmahanotmyname: yeah :)00:08
vinshmattoliverau: Where does that leave us then? :)00:08
vinshDon't want the segments container to have to be public...00:09
mattoliveraunotmyname: so *LO00:10
vinshHere is a summary I sent to the operators list today: http://lists.openstack.org/pipermail/openstack-operators/2015-December/009198.html00:10
mattoliveraunotmyname: so *LO's and ACLs, do segment containers have to be global? We can't passing referer along so can't lock em down.00:11
mattoliverauwow my english is awesome.00:11
notmynamemattoliverau: I left my +2 on the testr patch (the second +2). if you like it, can you also add a +2?00:11
notmynamemattoliverau: I'll read it upside down and it will make sense00:11
mattoliveraunotmyname: lol, thanks00:12
notmynamemattoliverau: or, some would say you don't speak english anyway ;-)00:12
notmynamemattoliverau: this is where acoles_ comes in to correct us both, right?00:12
mattoliverauI am one of them :P00:12
mattoliverauyup, exactly00:12
notmynamemattoliverau: do you think we should get acoles_ to also +2 the testr patch? or should you +A if you like it?00:12
notmynamehurricanerix: did you see the updates from gmmaha on patch 214206?00:13
patchbotnotmyname: https://review.openstack.org/#/c/214206/ - Modify functional tests to use ostestr/testr00:13
*** Jeffrey4l has joined #openstack-swift00:13
mattoliveraunotmyname: I'd like to +A, but I might have looked and been too involved in it now00:13
notmynamemattoliverau: vinsh: so yeah. *LO and permissions on the segments containers00:13
vinshIndeed, what's the deal with those?00:13
notmynamemattoliverau: meh. we're all involved. I just want to make sure we're all ok with the way tests are run now00:14
notmynamevinsh: basically, you've discovered how they work. reading your ops ML post now00:14
mattoliveraunotmyname: re: *LOs I wonder if we should add referer headers when making sub requests, so the egment containers can be locked down.. I can write a patch for that.. not that it will solve vinsh's question now :P00:15
notmynamemattoliverau: ah! it looks like there might be a bug in swift where the *LO middleware doesn't pass on the referrer (or the wsgi req.acl) for the subrequests00:15
vinshWhen we were in the joint swift-dev/swift-operators meeting in that smaller room in tokyo.  Ops were asked "whats tough about swift"  I faltered with an answer.. but meant to say.. crazy corner cases like this that users come up with :)00:16
notmynamemattoliverau: are we passing on the wsgi.authorize function? probably, since otherwise we have an info leak (authorized for DLO would allow access to everything00:16
mattoliveraunotmyname: I pretty sure we are.. I'll raise a bug, then investigate it. I've done enough debugging to know what referers isn't passed :)00:17
notmynamemattoliverau: cool, thanks00:18
vinshmattoliverau: cc/add me please "adam.vinsh"00:18
vinshThanks for considering this.00:18
notmynamevinsh: is that your Launchpad id?00:18
vinshoops id is "adam-vinsh"00:19
mattoliveraunotmyname: just wanted a confirmation that I wasn't going crazy and there wasn't an obvious reason a segment container needed to be global :P00:19
vinshditto00:19
notmynamemattoliverau: no, it should require the same permissions as the manifest00:19
mattoliverauthat's what I thought :)00:22
mattoliveraunotmyname: good news, I have found a replacement Aussie to send in my place to the mid-cycle, alex will go. Which means you need to make sure she is sarcastic and gives people a hard time in my wake :P00:26
notmynameoh, cool!00:26
asettleo/ hellooooo00:26
notmynameasettle: we'll be glad to see you!00:26
asettleI hope so! I was just going to see if we have set dates? :)00:27
mattoliveraualso means we can lock asettle and joel in a room to get swiftclient docs done :P00:27
asettleYeah, lucky Joel :P I'm going to email him nowish :)00:27
notmynametimburke should likely be there too00:27
notmynameasettle: cc me please :-)00:27
mattoliveraugreat!00:27
asettlenotmyname will do :)00:27
mattoliveraubut your all not allowed to have too much fun :P00:28
notmynameasettle: not that joe's doc patch landed, I'm going to see how I can get my outline proposed and landed too00:28
asettleOkay great :) looking forward to it.00:28
notmynamemattoliverau: you'll be having fun too. (look up the work meconium)00:29
notmyname*word00:29
asettleWOW WHY WOULD YOU DO THAT00:29
mattoliveraunotmyname: lol.. thanks :P00:29
notmynameit's not that bad. just part of having kids ;-)00:30
mattoliveraunotmyname: the baby is due around then, if I went to midcycle it'll come while away, now that I'm not it'll be very late.. cause that's how the world works :P00:30
notmynamemattoliverau: then it will have the same birthday as me ;-)00:31
asettleHaha yeah00:31
mattoliveraunotmyname: then notmyname will have to be here middlename00:32
mattoliveraus/here/her/00:32
*** david-lyle has joined #openstack-swift00:36
*** david-lyle_ has quit IRC00:36
timburkemattoliverau: don't listen to notmyname -- those really weren't so bad00:48
timburkewhat's bad is later, once she's on solids, and then she gets sick00:49
openstackgerritMerged openstack/python-swiftclient: Test 'string' behaviour of get_object  https://review.openstack.org/25801800:49
mattoliverautimburke: lol, thanks.. I'm going to pretend I didn't read that #ignoranceisbliss00:49
timburketotally worth it :)00:50
*** Jeffrey4l has quit IRC00:51
* mattoliverau is moving again, another doctors appointment, brb00:55
kota_good morning00:55
*** pgbridge has quit IRC00:56
*** zhill has quit IRC00:59
mattoliveraukota_: morning01:02
*** dmorita has quit IRC01:07
*** ho_ has joined #openstack-swift01:10
*** dmorita has joined #openstack-swift01:11
*** ho has quit IRC01:12
*** chlong has joined #openstack-swift01:21
kota_mattoliverau: o/01:24
*** zhill has joined #openstack-swift01:36
*** zhill has quit IRC01:40
*** klrmn has quit IRC01:42
*** lyrrad has quit IRC01:42
*** rvasilets__ has quit IRC01:45
*** ho has joined #openstack-swift01:47
*** ho_ has quit IRC01:48
janonymous_mattoliverau , kota_ : morning :)01:52
mattoliveraujanonymous_: o/01:52
janonymous_i see a lot of work on testr patch in mail, great work mattoliverau, gmmaha and others.01:55
mattoliveraujanonymous_: thanks, yeah, that took alot longer then expected, can now finally get back to other reviews :)01:55
*** klrmn has joined #openstack-swift01:55
*** km has quit IRC01:56
*** km has joined #openstack-swift01:56
*** zhill has joined #openstack-swift01:57
janonymous_janonymous_ : yeah i was in sync in initial commits but many changes from then till now.01:57
janonymous_*matoliverau01:57
janonymous_**mattoliverau01:57
*** noark9 has joined #openstack-swift01:59
notmynamejrichli: wbhuber: FYI http://openstacktexas.com02:01
*** zhill has quit IRC02:01
notmynamedfg: redbo: ^02:01
kota_janonymous: !02:02
kota_I'm now looking at your paste for Reconceiler unit test.02:03
*** klrmn has quit IRC02:04
janonymous_kota_:hey, thanks02:06
kota_janonymous: i can see that with direct nosetests so i could look at the reason02:06
janonymous_kota_: are you getting failure too02:07
kota_janonymous: right now, "tox -e py27" doesn't fail but "nosetests test/unit/container/test_reconciler.py" fails02:07
janonymous_yes02:07
kota_so probably there is a difference between runner environments.02:08
janonymous_kota_:yes maybe.02:08
kota_i'm thinking that comes from... time zone?02:08
kota_not sure02:08
kota_that is because non-US people seeing it02:09
janonymous_try tox -e py27 test.unit.container.test_reconciler02:09
janonymous_that fails too02:09
kota_oh, really02:09
kota_will try02:09
kota_done and looks like reproduced02:11
janonymous_:)02:11
kota_but to be able to reproduce is a way to fix it :-)02:11
janonymous_hehee yes02:12
* janonymous_ travelling to work02:12
janonymous_kota_ :will be back in 30 mins :)02:13
asettlenotmyname you still awake?02:14
*** klrmn has joined #openstack-swift02:15
*** janonymous_ has quit IRC02:17
kota_janonymous: fyi, it worked well in environ running on UTC02:19
ZyricHi all. I've installed Slogging and left it going for a few hours today but the only logs I have are access ones. Looking at the logs all the account/container stats have 404'd for some reason. Thoughts?02:34
*** Jeffrey4l has joined #openstack-swift02:37
*** mac_ified has joined #openstack-swift02:38
*** zhill has joined #openstack-swift02:40
*** gyee has quit IRC02:41
*** zhill has quit IRC02:45
*** sanchitmalhotra has joined #openstack-swift02:49
kota_janonymous: fixed, will push patch soon.02:49
janonymouskota_: great02:58
openstackgerritKota Tsuyuzaki proposed openstack/swift: Fix reconciler test to calc lastmodified as UTC  https://review.openstack.org/25823302:59
kota_done02:59
kota_janonymous: please try it :)02:59
janonymouskota_: i was just fixing ut issues for testr patch , i got these errors then :)03:00
janonymouskota_: i saw the bug with fix . Yes i just tested it , and it passes . yay :)03:03
*** zhill has joined #openstack-swift03:21
*** venkat has joined #openstack-swift03:25
kota_janonymous: yey :)03:25
*** zhill has quit IRC03:26
kota_Zyric: did you set up stats-logger and log-uploader?03:27
kota_they are tools to collect account/container stats03:28
Zyrickota_: I suppose I have, I followed the guide at https://github.com/notmyname/slogging/blob/master/doc/source/overview_stats.rst03:29
ZyricI have one of the storage.error files (number four I think) paste-binned here: http://paste.openstack.org/show/482035/03:30
kota_Ah, it looks not a slogging error but just Swift, right?03:32
kota_Have you create AUTH_test account for your Swift?03:33
mattoliverausigh, been playing with a patch to fix referers on segment containers.. easy to fix, but it wasn't working and debugged the hell out of it.. just to find I mistyped the  referrer (.r <url>) on the segments container. #FAIL03:35
mattoliveraubeen one of those afternoons03:35
Zyrickota_: Auth_test is the default test account, I believe03:36
*** nadeem has joined #openstack-swift03:37
kota_Zyric: Did the test work well?03:37
Zyrickota_: When I enter a get command on the account it appears to work03:38
kota_Curious, swift logged as the account was not there but worked well....03:39
Zyrickota_: Yes, I can GET /v1/AUTH_test/ using curl -v -H 'X-Auth-Token: <authToken>' http://127.0.0.1:8080/v1/AUTH_test/ and it doesn't 404 me03:40
*** dmorita has quit IRC03:46
*** dmorita has joined #openstack-swift03:47
kota_sorry not sure. but what i can suggest is that perhaps you need to make sure the swift setup configuration and find it to trace correct route (from proxy to account server)03:50
Zyrickota_: Hm alright, so check the swift configuration files for information? Like swift.conf and /etc/swift/object-server.conf and such?03:53
kota_yup and espically /etc/swift/account.ring.gz03:54
kota_swift-get-nodes might help you03:55
kota_that's a binary command to find the node devices from your supeficied path and ring information.03:56
ZyricNice, thanks for the information kota_ I'll look into those03:57
ZyricNot sure how Slogging is getting confused with pathing considering the only locational data I set is the account name for uploading logging data.03:58
*** km_ has joined #openstack-swift04:02
*** km has quit IRC04:04
*** kei_yama has quit IRC04:04
*** kei_yama has joined #openstack-swift04:13
venkatHi all,04:13
venkatOne doubt : md5 is not collision resistant.04:13
venkatI am taking examples for creating containers.04:14
venkatHow can we ensure that every new PUT request to container, creates a new database file with04:15
venkathexdigest as filename04:15
*** asettle is now known as asettle-afk04:19
*** proteusguy_ has quit IRC04:20
notmynameasettle-afk: I'm here04:26
venkatCan someone can answer the above question?04:27
notmynamevenkat: all hash functions have collisions (by definition). the question is what is the chance of a hash collision given the inputs04:29
notmynamevenkat: and while md5 is used for the placement (and the resulting filenames), the probability of collision (especially when looking at eg 2nd preimage attacks) has been seen as very low04:31
notmynamemeaning, sure, it could be changed, and probably will at some point, but it's not something that is causing issues such that those running swift are prioritizing that change (including those with billions of objects)04:31
notmynamevenkat: and I'm not trying to dismiss the concerns you have about md5!04:32
venkatnotmyname: Ok, Thanks, sha-1 is better than md5 right?04:33
*** proteusguy_ has joined #openstack-swift04:33
notmynamevenkat: yes, but wise counsel would say sha-1 shouldn't be used for security either04:33
notmynamevenkat: for some specifics on the implementation, here's what is hashed (with md5) in swift for placement and on-disk filename04:34
venkatnotmyname: Ok, Thnak you04:34
venkat*Thank04:35
notmynameprefix + / + account + / + container + / + object + / + suffix04:35
notmynameyou (the user) control account, container, and object strings04:35
notmynameand the total of those three (plus the slashes between them) needs to be less than 1536 bytes, after utf8 encoding and url quoting04:36
notmyname1536 = 256 + 256 + 102404:37
notmyname256 being the default max size for account and container04:37
notmynameand 1024 being the default max size for object name04:38
notmynameoh, and while you sortof maybe in some situations might have control over the object name, you really don't have much control over it. unless you own the cluster, you have to use what you're given for the account part from the cluster04:38
notmynameso really, you the user only control the container and object part04:39
notmynameso yeah, 256 + 1024 utf8/quoted bytes is much bigger than the 2**128 bits in md5 (so by definition there are collisions there)04:40
venkatnotmyname: Thanks, this information is very helpful..04:41
notmynamevenkat: questions for you, now :-)04:42
notmynamevenkat: why do you ask? are you being asked about it?04:42
notmynamevenkat: do you have a swift cluster where this is an issue?04:42
venkatI am going through the code, I understood how url's are mapped to hexdigest.04:43
venkatlater I explored md504:43
venkatIn the wiki, MD5 is not collision resistant.04:44
*** badari has quit IRC04:44
notmynameright04:44
venkatin the future MD5 should no longer be implemented...where a collision-resistant hash function is required.04:44
venkatabove line is from wiki04:45
notmynamewhat wiki?04:45
venkathttps://en.wikipedia.org/wiki/MD5#Collision_vulnerabilities04:45
notmynamevenkat: so why are you perusing the swift code? ;-)04:46
notmynameZyric: did you get your questions answered?04:48
venkatI am learning swift code, just I am interested in open-source projects04:49
*** trifon has joined #openstack-swift04:49
notmynamevenkat: awesome! I'd love to help you get involved04:49
notmynamewe've got lots of stuff to do, and lots of ways to help out04:50
venkatnotmyname: Ok, Thank you, sure i will help04:50
notmynamevenkat: so you're already doing 2 things which are super important for anyone contributing to swift. (1) asking questions and being in this IRC channel and (2) looking at the swift code to see what it does04:51
venkatok04:52
notmynamevenkat: if you look in the channel topic, there's a like to a review dashboard. it's a good place to see what changes people are suggesting for swift, and it's our code review tool04:52
notmynameand our bug tracker is at https://bugs.launchpad.net/swift (for swift) and https://bugs.launchpad.net/python-swiftclient (for the official SDK/CLI)04:53
notmynamevenkat: are you familiar with openstack projects? have you contributed to any others?04:53
notmynameon the launchpad bug trackers, the "wishlist" items are things we think would be cool to add to the project04:53
venkatnotmyname: yes, I worked on cloudkitty-billing project, I submitted 1 patch to nova04:53
notmynameoh, cool04:54
notmynameso what brings you to swift? have you deployed it? used it? just really really like storage?04:54
venkatI tried only SAIO and devstack04:54
notmynamevenkat: oh cool! now you've doing something else that's good: you've set up the SAIO :-)04:55
venkatOk, I have the SAIO setup04:56
*** badari has joined #openstack-swift04:56
notmynamevenkat: any chance you work for hgst?04:57
venkatyes, how you know?04:58
notmynamelol. /me is sneaky04:59
notmynameyou're connected to IRC from an IP address provisioned to hgst05:00
venkatOk, you are too smart05:00
notmynamevenkat: so anyway... is there something in swift that you're interested in? something way I can help you get involved?05:01
notmynamevenkat: so as to have parity in the information sharing, I'm http://not.mn05:01
*** kei_yama has quit IRC05:02
venkatYes, I want to know how db's are replicated across the conatiner servers05:02
*** km has joined #openstack-swift05:02
venkatwhen the background replication occurs?05:02
notmynamevenkat: http://docs.openstack.org/developer/swift/overview_replication.html#db-replication05:02
notmynameyes05:02
*** km_ has quit IRC05:03
*** ppai has joined #openstack-swift05:03
venkatnotmyname: Thank you, I will go through the above link05:08
venkatmy github link :     https://github.com/venkatesh55105:08
*** dmorita has quit IRC05:08
notmynamevenkat: please feel free to drop by and ask questions as you have them. please feel free to leave reviews on patches, and if you've got some code to share, please feel free to submit it!05:10
*** dmorita has joined #openstack-swift05:11
*** SkyRocknRoll has joined #openstack-swift05:11
*** dmorita has quit IRC05:15
*** yatin has joined #openstack-swift05:20
*** zhill has joined #openstack-swift05:20
*** badari has quit IRC05:24
*** kei_yama has joined #openstack-swift05:25
venkatnotmyname: sure, I am very much interested in open-source contribution.05:26
*** trifon has quit IRC05:31
openstackgerritvenkatesh proposed openstack/swift: remove unused parameters in the method  https://review.openstack.org/24340105:31
notmynamevenkat: oh wait. is that you? ^05:31
venkatnotmyname: yes05:33
*** links has joined #openstack-swift05:34
*** klrmn has quit IRC05:34
*** SkyRocknRoll has quit IRC05:37
*** zhill has quit IRC05:40
*** SkyRocknRoll has joined #openstack-swift05:45
*** lpabon has quit IRC05:45
*** SkyRocknRoll has quit IRC06:12
*** SkyRocknRoll has joined #openstack-swift06:26
openstackgerritMatthew Oliver proposed openstack/swift: Pass HTTP_REFERER down to subrequests  https://review.openstack.org/25828006:31
*** changbl has joined #openstack-swift06:42
*** asettle-afk is now known as asettle06:44
asettleoof notmyname are you still around? sorry! Ran off to the docs06:44
*** nakagawamsa has joined #openstack-swift06:51
*** nakagawamsa has quit IRC06:53
*** nakagawamsa has joined #openstack-swift06:58
*** bsdkurt has quit IRC07:00
*** rcernin has joined #openstack-swift07:01
*** bsdkurt has joined #openstack-swift07:02
*** km_ has joined #openstack-swift07:08
*** km has quit IRC07:08
*** zaitcev has quit IRC07:08
*** nadeem has quit IRC07:09
*** rcernin_ has joined #openstack-swift07:12
*** wanghua has joined #openstack-swift07:12
*** rcernin has quit IRC07:13
wanghuaHi all, does swift have the same policy mechanism which can be configured in policy.json as other openstack projects?07:14
*** bsdkurt has quit IRC07:15
*** bsdkurt has joined #openstack-swift07:17
*** rcernin_ has quit IRC07:21
*** rcernin has joined #openstack-swift07:22
openstackgerritMerged openstack/swift: Modify functional tests to use ostestr/testr  https://review.openstack.org/21420607:25
*** Jeffrey4l has quit IRC07:31
*** ntt has joined #openstack-swift07:35
*** Jeffrey4l has joined #openstack-swift07:44
*** arnox has joined #openstack-swift07:58
janonymouskota_ : just one ques, why was it passing when all ut were run, why doesn't it fail ?08:00
*** torgomatic has quit IRC08:01
kota_janonymous: sorry, idk08:02
kota_right now08:02
kota_perhaps some codes would exist to adjust default timezone to utc?08:04
kota_not sure.08:04
*** Jeffrey4l has quit IRC08:08
*** chlong has quit IRC08:17
*** Jeffrey4l has joined #openstack-swift08:21
*** hseipp has joined #openstack-swift08:24
*** Jeffrey4l has quit IRC08:26
*** ppai has quit IRC08:36
*** rvasilets__ has joined #openstack-swift08:36
*** Jeffrey4l has joined #openstack-swift08:38
*** zhill has joined #openstack-swift08:39
*** jordanP has joined #openstack-swift08:44
*** zhill has quit IRC08:45
*** ppai has joined #openstack-swift08:50
*** rcernin has quit IRC08:50
*** Jeffrey4l has quit IRC08:55
*** mac_ified has quit IRC08:58
*** Jeffrey4l has joined #openstack-swift08:59
kota_k, heading for home.09:02
kota_bye09:02
*** venkat has quit IRC09:05
cschwedevinsh: thx for the info on my puppet-swift patch - i just rebased that one. let me know if this looks ok to you in the review09:05
*** rcernin has joined #openstack-swift09:08
*** Jeffrey4l has quit IRC09:11
*** jistr has joined #openstack-swift09:13
janonymouskota_: thanks :)09:15
*** venkat has joined #openstack-swift09:19
*** noark9 has quit IRC09:19
*** jmccarthy1 has joined #openstack-swift09:39
*** jmccarthy1 has quit IRC09:39
*** jmccarthy1 has joined #openstack-swift09:40
*** joeljwright has joined #openstack-swift09:40
*** ChanServ sets mode: +v joeljwright09:40
*** jmccarthy1 has quit IRC09:40
*** venkat has quit IRC09:46
*** acoles_ is now known as acoles09:47
*** venkat has joined #openstack-swift09:58
*** jmccarthy has quit IRC09:59
*** jmccarthy has joined #openstack-swift10:00
*** Jeffrey4l has joined #openstack-swift10:04
eranromacoles here?10:04
*** rledisez has joined #openstack-swift10:07
acoleseranrom: yes, hi10:08
acoleswow long scrollback today10:09
acolesmattoliverau: i thought i read that notmyname was going to become middleware, but then saw middlename :)10:09
acolesmattoliverau: which got me thinking that you could use the swift canonical pipeline as middlenames...10:10
acolesok so the func test patch landed, time to go play with it...10:11
mattoliverauacoles: lol genius!10:13
acolesmattoliverau: hmm, "genius" is a good name too10:13
*** Jeffrey4l has quit IRC10:15
*** Jeffrey4l has joined #openstack-swift10:15
mattoliverauSo is Alistair ;)10:16
acolesno no no10:17
openstackgerritHisashi Osanai proposed openstack/swift: Fix duplication for headers in Access-Control-Expose-Headers  https://review.openstack.org/25839210:17
acolesso prone to mispeling10:17
acoles(sic)10:17
*** jmccarthy1 has joined #openstack-swift10:20
tdasilvaacoles: isn't there a famous "middle..." something or other in the UK? ;)10:25
acolestdasilva: middleton?10:26
tdasilvaif you adtop Acoles Genius Middleware you could almost pass as a royalty10:26
tdasilvaswift royalty10:27
tdasilvaduh, Alistair .....10:27
acolestdasilva: lol10:29
acolestdasilva: mattoliverau : TypeError: unbound method test_bad_metadata2() must be called with TestAccount instance as first argument (got nothing instead)10:30
*** aix has joined #openstack-swift10:30
acoles:/10:30
acolesgot that using: tox -r -e func -- -n test.functional.test_account.TestAccount.test_bad_metadata210:30
mattoliverauWhat :( it worked on mine, John's and tdasilva's10:31
acolesmattoliverau: but that was *yesterday* :P10:31
tdasilvaacoles: instead of -n, use -r for regex10:31
mattoliverauLol10:31
tdasilvatox -e func -- -r  test.functional.test_account.TestAccount.test_bad_metadata210:31
acolestdasilva: yep that works, but in scrollback i thought mtreinish was saying -n should work10:33
acolesthis works: tox -r -e func -- -n test.functional.test_account.TestAccount10:34
acolesso must be that -n is only happy with a class, not a test method10:34
acolesmattoliverau: did you see -n work with a method, or just class?10:35
tdasilvaacoles: I never got -n to work with a method, only class10:35
eranromacoles: so I am struggling with the so called redundant call to get_container to allow the changes to get commited.10:36
mattoliverauYeah only class in the end for me to. But -r works to the class so that's good10:36
eranromthe code basically does: put_container (that is synced), put_object, get_container, and then run-once the sunc daemon10:37
eranromwithout the get_container between the put_object and the run-once the broker just do not show the new object10:38
eranromSo I went and printed whatever gets merged as a result of get_info after each of the steps.10:38
eranromI consistently get:10:39
*** ho has quit IRC10:39
eranromAfter the put_object I see: Going to merge items: [{'object_count': '0', 'name': 'dest-container-414d74d3-4904-479d-a86f-4f473f1ff1ce', 'deleted': 0, 'bytes_used': '0', 'put_timestamp': '1450259790.01652', 'storage_policy_index': '0', 'delete_timestamp': '0'}]10:40
eranromAfter the get_container I see: Going to merge items: [{'name': 'object-50567314-f745-41fa-bfdc-77442a98c7dd', 'deleted': 0, 'created_at': '1450259790.13798', 'storage_policy_index': 1, 'etag': '70b68ae721a61941a1a62724dde5d5e4', 'content_type': 'application/octet-stream', 'size': 9}]10:40
eranromthe latter is not seen without the get_container. FWIW and not surprisingly head_container has the same effect10:41
acolestdasilva: mattoliverau this is fun http://paste.openstack.org/show/482053/10:41
acolesgmmaha: ^^10:41
eranromLet me dig dipper...10:42
tdasilvaacoles: func: commands succeeded :/10:43
acolestdasilva: yeah i love that positivity :)10:44
acoleseranrom: ok, so you only see the object update flushed to the db during the get_container, but not when sync calls get_info?10:45
eranromacoles: correct10:46
eranromCan object upload return with async pending from all object nodes?10:46
acoleseranrom: yes - if you mean can the object PUT return before any container has been updated10:47
eranromyes, in fact before any container replica has been updated.10:48
acoleseranrom: at least i believe that to be so, there was a recent change to how object->container updates are handled10:48
acolesbut that just made them potentially more likely to be async10:48
acoleseranrom: so could it just be that the object update has not arrived at container server when you log the get_info call but has arrived when the get_container occurs?10:49
eranromyes, I think that that's is what hapenning. so the call is redundant, and it just a matter of waiting.10:50
eranromthat is the call to get_container10:50
eranromI will try10:51
acoleseranrom: your first "Going to merge items" log above is a container->account update, the second is an object->container update10:51
eranromRight. So the first one is triggered by the creation of the container10:53
acolestdasilva: i'll file a bug report for the --until-failure problem, looks like the testr.conf command may need working on.10:53
eranromand the second is by the objecyt upload10:53
tdasilvaacoles: ok, i was about to test it here on my end10:53
tdasilvaacoles: sorry i forgot to test that, i hadn't used the flag before10:54
tdasilvait actually sounds very useful10:54
acolestdasilva: yes, although what I will really miss is nosetests -x, which runs tests once stopping on first failure10:55
tdasilvaacoles: oh, i thought that's what --until-failure did10:55
acolestdasilva: well maybe, I don't know yet :), but the os-testr docs say it *repeats* the tests until failure10:56
tdasilvammm...that sounds weird10:56
*** Jeffrey4l has quit IRC10:58
*** aix has quit IRC10:59
*** aix has joined #openstack-swift11:00
acolestdasilva: bug 152672511:00
openstackbug 1526725 in OpenStack Object Storage (swift) "tox -e func -- --until-failures does not work" [Undecided,New] https://launchpad.net/bugs/152672511:00
tdasilvaacoles: thanks11:04
acolestdasilva: I can of course still use nosetests -x but not via tox, so its not so bad.11:06
tdasilvaacoles: true true11:06
tdasilvaI was about to test removing the env. variable to see if that clears up the error11:07
acolestdasilva: maybe we could have tox -e func_with_nose :) like the good ol' days11:07
tdasilvaha!11:07
tdasilvatox -e func_old_school11:07
acolesi keep laughing at clayg's comment recently 'every release is a little bit better than the next'11:08
acolestdasilva: thats even better11:08
acoleseranrom: the object server will wait 1 sec for container update to complete swift/obj/server.py:30411:10
acoleseranrom: but the thread will wait 3 secs before timing out on the container server request and queueing the update for async re-try11:10
acoleskota_: thanks for re-review11:12
acoleskota_ is probably out partying at this time :)11:13
kota_Not so a frequent party!11:14
kota_You are welcome, acoles :)11:14
*** joeljwright has quit IRC11:15
eranromacoles: Thanks. Apparently, the commit_puts_stale_ok call triggered from get_info from container sync never gets to aquire the lock over the pending flie.11:17
eranromI will dig more into this.11:17
eranromgotta go now11:17
eranromThanks!11:17
tdasilvamtreinish: are you aware of issues with --until-failure? We getting errors running like this: tox -e func -- --until-failure11:29
tdasilvalooks like neutron also has similar issue: https://bugs.launchpad.net/os-testr/+bug/146720911:30
openstackLaunchpad bug 1467209 in os-testr "--until-failure option does not work" [Undecided,New]11:30
nttHi, How the swift proxy point to the object/container and account server? I cannot find any reference in /etc/swift/proxy-server.conf?11:36
tdasilvantt: that information comes from the rings11:37
nttso, the proxy parse the ring file and obtains ip of object/container/account server?11:37
tdasilvathe proxy gets that information from the Ring class.11:40
tdasilvantt: checkout some of these links: http://docs.openstack.org/developer/swift/overview_architecture.html11:40
tdasilvahttps://swiftstack.com/openstack-swift11:40
tdasilvantt: this is also helpful: http://www.florentflament.com/blog/openstack-swift-ring-made-understandable.html11:41
ntttdasilva: thank you11:42
*** rvasilets__ has quit IRC11:45
*** goodygum has joined #openstack-swift11:51
openstackgerritjanonymous proposed openstack/swift: Fix reconciler test to calc lastmodified as UTC  https://review.openstack.org/25823311:53
*** yatin has quit IRC11:55
*** km_ has quit IRC12:06
*** kei_yama has quit IRC12:10
*** CaioBren_ has joined #openstack-swift12:20
*** CaioBre__ has joined #openstack-swift12:22
*** CaioBren_ has quit IRC12:22
*** CaioBrentano has quit IRC12:23
acoleseranrom: i think i see the cause of needing the get_container call. the get_info only commits from pending file if pending file exists, and there is no pending file in the sync_containers sub_dir, which is where the sync daemon thinks the db is and therefore where it looks for the pending file.12:30
*** joeljwright has joined #openstack-swift12:30
*** ChanServ sets mode: +v joeljwright12:30
acoleseranrom: a very quick hack...http://paste.openstack.org/show/482058/12:30
acoleseranrom: left comment on gerrit12:37
kota_back12:38
*** CaioBre__ has quit IRC12:47
eranromacoles: Thanks! Great catch! I think the right way to do this is to initiate the broker in container sync from the pointed dir rather then the symlink. I wil try that later and fix accordingly.12:52
*** links has quit IRC12:57
*** rcernin has quit IRC12:58
*** lpabon has joined #openstack-swift13:00
*** CaioBrentano has joined #openstack-swift13:06
*** chlong has joined #openstack-swift13:10
*** rcernin has joined #openstack-swift13:13
*** Jeffrey4l has joined #openstack-swift13:16
*** ppai has quit IRC13:21
vinshcschwede: for the auditor block.. copy the replicator block above and just change replicator to auditor13:22
vinshIt should be the same form also using the service class.13:23
*** jmccarthy1 has quit IRC13:24
cschwedevinsh: thx - you mean in manifests/storage/generic.pp, right?13:25
*** breitz has quit IRC13:32
*** zhill has joined #openstack-swift13:32
vinshYep!13:32
*** breitz has joined #openstack-swift13:32
*** zhill has quit IRC13:37
vinshcschwede: One more thing to.. in the spec file... copy the replicator block also.. will preserve the "ensure" enable/disable testing in the spec :)  needs name passed in all that the same.13:37
vinshThen, I say its ready to go :)  I'm glad to see this merge soon... I'm using it as a starting point for my next review.  I'm adding support for a dedicated replication network in it.13:38
*** SkyRocknRoll has quit IRC13:43
kota_someone could have a time to review my too small 2 patches?13:45
kota_patch 258233 and patch 25509613:46
patchbotkota_: https://review.openstack.org/#/c/258233/ - Fix reconciler test to calc lastmodified as UTC13:46
patchbotkota_: https://review.openstack.org/#/c/255096/ - Sleep enough for trampoling13:46
kota_one is 1 line change and the other is 2 lines :)13:47
janonymousboth fixes UT for me :)13:47
kota_janonymous: me too ;)13:48
*** delatte has joined #openstack-swift13:48
*** venkat has quit IRC13:51
*** delattec has quit IRC13:52
*** lcurtis has joined #openstack-swift14:02
*** hseipp has quit IRC14:04
*** rcernin has quit IRC14:06
gmmahaacoles: tdasilva: i didnt try that piece out. Let me check it out14:08
*** eranrom has quit IRC14:08
acoleskota_: hehe i fixed one of those a year ago in fast-post patch :) https://review.openstack.org/#/c/135380/19/test/unit/container/test_reconciler.py14:08
*** delattec has joined #openstack-swift14:09
*** ekarlso has quit IRC14:10
kota_acoles: great! ...not landed yet! :/14:10
acoleskota_: i think your patch will land before mine ;)14:11
kota_acoles: lol14:13
*** delatte has quit IRC14:13
*** changbl has quit IRC14:14
*** ctennis has quit IRC14:16
*** ctennis has joined #openstack-swift14:16
lcurtisgood morning all14:17
lcurtisdoes anyone know if it is possible to increase xfs inode cache size beyond 32gb?14:17
gmmahaacoles: tdasilva: merging the .testr.conf makes it go fine, but hitting this error now http://paste.openstack.org/show/482076/14:18
gmmahahttp://paste.openstack.org/show/482077/14:19
*** daemontool has joined #openstack-swift14:19
tdasilvagmmaha: :/ it seems to be like there's an issue with testr14:21
gmmahatdasilva: yeah.. seems like it.14:22
gmmahawondering if there are more configs that we need/should set to make the --until-failure work14:23
acolesgmmaha: what did you mean by " merging the .testr.conf" ?14:25
acolesgmmaha: oic your second paste14:26
gmmahaacoles: yeah, not breaking up the line14:33
acolesgmmaha: iirc this http://paste.openstack.org/show/482076/ is what i saw when trying an earlier patchset14:33
gmmahaacoles: yeah, that error seems familiar. think its close to the same we are seeing when you run the test with -n and give it a function against a class14:35
*** blmartin has joined #openstack-swift14:42
pdardeaugood morning14:44
gmmahagood morning pdardeau14:48
wbhubernotmyname: Openstack TX sounds great.  I'm bookmarking it.14:49
*** joeljwright has quit IRC14:49
pdardeaugmmaha: Guten tag!14:49
*** joeljwright has joined #openstack-swift14:51
*** badari has joined #openstack-swift14:51
*** ChanServ sets mode: +v joeljwright14:51
*** dustins|out is now known as dustins14:52
*** daemontool_ has joined #openstack-swift14:54
*** joeljwright has quit IRC14:55
*** daemontool has quit IRC14:56
*** joeljwright has joined #openstack-swift14:58
*** ChanServ sets mode: +v joeljwright14:58
*** daemontool_ has quit IRC14:59
acolesgmmaha: got it! our functional test classes subclass unittest.TestCase, changing to unittest2.TestCase and --until-failures and -n both work (along with test_command being on  single line)15:09
acolestdasilva: mattoliverau ^^15:09
gmmahaacoles: ohhh15:10
acolesgmmaha: I am just doing global replace to check complete test suite15:10
gmmahano way i would have figured that out. :) thanks a ton!15:10
tdasilvaneat15:10
gmmahaacoles: awesome15:10
acolesgmmaha: i'll push a patch and then everyone can try it out15:10
gmmahaabsolutely15:11
tdasilvaacoles: thanks!15:11
*** jmccarthy1 has joined #openstack-swift15:11
*** jmccarthy1 has quit IRC15:14
gmmahaacoles: the unittest2 page says its a backport of unittest features from py2.7 and onwards https://pypi.python.org/pypi/unittest215:16
*** dmorita has joined #openstack-swift15:16
*** jmccarthy has quit IRC15:16
gmmahashould we look into updating theunittest package also?15:16
acolesgmmaha: sorry, updating where?15:20
gmmahaacoles: ignore that comment.. i thought there were two seperate packages unittest and unittest215:21
gmmahadoesnt seem like it..15:21
acolesargh, now our skipped tests show as fails!15:22
*** tsg has joined #openstack-swift15:26
*** zaitcev has joined #openstack-swift15:29
*** ChanServ sets mode: +v zaitcev15:29
openstackgerritAlistair Coles proposed openstack/swift: Fix func test --until-failure and --no-discover options  https://review.openstack.org/25857815:35
acolesgmmaha: i included your fix to .testr.conf and made you co-author ^^15:36
gmmahaacoles: thanks.. :)15:37
* gmmaha goes to test the patch15:37
acolesmtreinish: would you mind taking a look at this patch  https://review.openstack.org/258578 ? It seems to work but I got there by trial and error rather than great understanding!15:38
*** daemontool has joined #openstack-swift15:40
*** acoles is now known as acoles_15:43
gmmahaacoles_: are you also seeing a setup and teardown for each test with --until-failure15:47
gmmaha?15:47
mtreinishacoles_: oh, cool. Yeah nose isn't really unittest compatible. We hit that same thing before when we did the initial tempest migration a few years ago15:48
*** pgbridge has joined #openstack-swift15:59
*** acoles_ is now known as acoles15:59
acolesmtreinish: thanks16:01
acolesgmmaha: yes, --until-failure seems to invoke the package setup/teardown for every test method, and with that comes a lot of noise16:01
acolesbut at least it works!16:02
gmmahaacoles: ohh yeah.. just wanted to verify that the outcome is consistent16:04
*** joeljwright has quit IRC16:04
*** joeljwright has joined #openstack-swift16:09
*** ChanServ sets mode: +v joeljwright16:09
*** joeljwright has quit IRC16:14
*** ekarlso has joined #openstack-swift16:17
*** janonymous_ has joined #openstack-swift16:19
*** daemontool__ has joined #openstack-swift16:21
-openstackstatus- NOTICE: Gerrit will be offline for a software upgrade from 17:00 to 21:00 UTC. See: http://lists.openstack.org/pipermail/openstack-dev/2015-December/081037.html16:21
*** daemontool has quit IRC16:21
*** janonymous_ has quit IRC16:22
*** janonymous_ has joined #openstack-swift16:23
*** garthb has joined #openstack-swift16:25
*** zhill has joined #openstack-swift16:30
*** minwoob has joined #openstack-swift16:33
*** alejandrito has joined #openstack-swift16:44
*** esker has joined #openstack-swift16:48
notmynamegood morning16:52
notmynameI'll be in and out today. tons of meetings (including the swift team meeting)16:52
janonymous_notmyname: o/16:54
*** daemontool__ has quit IRC16:56
janonymous_I have added a patch for opinions in https://wiki.openstack.org/wiki/Meetings/Swift, as suggested by Ondrej in the comments.16:57
janonymous_notmyname: But it gets pretty late for me to attend it, should i remove it from there or is it ok ?16:57
notmynamejanonymous_: do you have a specific question that you want addressed? eg "shall we do X or Y in this case"16:58
janonymous_Oh, not particularly , it about how should we proceed with config values in config files and code16:59
janonymous_*it's16:59
*** diazjf has joined #openstack-swift17:00
notmynamejanonymous_: ok. we've got a lot on the agenda already, so I think it would be good to keep a discussion in gerrit. I can leave the patch reference on the agenda, though, to ask for some reviews on it17:01
notmynamejanonymous_: if you've got a specific question, perhaps asking it in gerrit would be a good async way for people to respond17:01
*** lcurtis has quit IRC17:02
notmynamejanonymous_: also that means you don't have to be up ridiculously early in the morning :-)17:02
janonymous_yes, but i was suggested to ask it in meeting if possible in gerrit :)17:02
janonymous_*gerrit comment17:03
*** daemontool__ has joined #openstack-swift17:03
*** arnox has quit IRC17:03
notmynameseems the questions are from onovy17:03
janonymous_yup17:04
onovy?17:05
janonymous_it would be good if someone could review and land some opinions on the same, then no need to land it in meeting i guess17:05
janonymous_onovy: o/17:05
notmynamejanonymous_: agreed17:05
janonymous_onovy: its about patch https://review.openstack.org/#/c/227855/17:06
-openstackstatus- NOTICE: Gerrit is offline for a software upgrade from 17:00 to 21:00 UTC. See: http://lists.openstack.org/pipermail/openstack-dev/2015-December/081037.html17:06
*** ChanServ changes topic to "Gerrit is offline for a software upgrade from 17:00 to 21:00 UTC. See: http://lists.openstack.org/pipermail/openstack-dev/2015-December/081037.html"17:06
*** lcurtis has joined #openstack-swift17:06
onovyit's long open review so i suggested to discuss it on meeting17:07
onovymy opinion is in gerrit17:07
*** janonymous__ has joined #openstack-swift17:08
janonymous_onovy: i also agree with what you suggested17:09
janonymous_onovy: So, i think i would make the changes as per that and some review comments would do the good17:10
janonymous_onovy: what do you say.. ?17:10
*** rledisez has quit IRC17:10
*** daemontool__ has quit IRC17:12
tdasilvais gerrit down for anybody else?17:12
diazjftdasilva -> http://lists.openstack.org/pipermail/openstack-dev/2015-December/081037.html17:13
notmynameyup17:13
notmynamesee the topic17:13
*** lcurtis has quit IRC17:14
onovyjanonymous_, yep, that's good idea17:14
notmynametdasilva: this would be a good time to peruse https://github.com/google/git-appraise ;-)17:14
tdasilvaha! thanks17:14
tdasilvai think I saw that on hacker news the other day17:15
notmynameyeah, that's where I saw it17:15
notmynamelooked really interesting17:15
notmynameand I still think we should have something like gerrit that is an interface to it. just not gerrit as the system which must be online for anyone to do anything17:15
tdasilvamaybe we can just do it here17:15
tdasilvaacoles: your patch works, +2 :D17:15
notmynameheh17:16
gmmahaacoles: +1 from me on the patch. it works for me. Just cant put it in gerrit now :)17:16
janonymous_Haha (Y)17:16
tdasilvaacoles: I also tried --pdb and it now works with the specific test case, so great work, thanks for digging into it17:16
*** zhill has quit IRC17:17
tdasilvanotmyname: so..just wondering...what's the next step regarding the tempest tests?17:17
janonymous_Okay cool17:17
janonymous_onovy: thanks17:18
acolesgmmaha: tdasilva: great17:18
notmynametdasilva: next step, I think, is to talk to the tempest team (ie mtreinish) and let them know and hopefully they can pull our tests in now17:18
janonymous_btw, i was working with testr for unittests ,it's nearly done with some ut's left for proxy to fix.17:20
acolesif gerrit is down i guess i will have to do expense reports17:20
*** daemontool__ has joined #openstack-swift17:25
*** lyrrad has joined #openstack-swift17:25
janonymous_notmyname: Thanks, i can go now :)17:25
janonymous_see ya guys. bye17:25
notmynamegood night17:25
*** gyee has joined #openstack-swift17:26
*** janonymous__ has quit IRC17:27
*** janonymous_ has quit IRC17:27
pchngIs review.openstack.org down for anyone else? I'm getting a 503 error17:29
gmmahapchng: i am told there is a known maintanence happening17:29
pchnggmmaha: okay, thanks for the note17:29
gmmahahttp://lists.openstack.org/pipermail/openstack-dev/2015-December/081037.html17:29
pchngah, okay, thanks explains it :)17:30
gmmahapchng: anytime17:30
*** Jeffrey4l has quit IRC17:31
*** zhill has joined #openstack-swift17:37
*** zhill has quit IRC17:42
*** jistr has quit IRC17:42
*** daemontool__ has quit IRC17:43
claygheyoh!17:43
*** diazjf has quit IRC17:43
*** daemontool__ has joined #openstack-swift17:44
pdardeauhey clayg!17:47
tdasilvaclayg: hello!17:47
acolesclayg: hi17:48
clayghey fellas17:50
claygI'm digging out of emails and reading diffs today - i'd like to get a handle on what we have lined up with priority reviews asap17:50
*** daemontool__ has quit IRC17:51
*** logan- has quit IRC17:51
acolesclayg: eranrom and I made some progress with his container sync patch 205803, he's making a change right now i think17:53
patchbotacoles: https://review.openstack.org/#/c/205803/17:53
acolesclayg: the func test change landed17:53
*** dabukalam has quit IRC17:54
acolesclayg: kota has a +2 on patch 23112117:54
patchbotacoles: https://review.openstack.org/#/c/231121/17:54
acolesdidn't patchbot used to give us the commit subject?17:54
acolesmaybe it can't while gerrit is down17:54
*** dabukalam has joined #openstack-swift17:54
claygoh, sweet - no gerrit should make it easier to work on emails ;)17:55
acolestrue :)17:56
*** diazjf has joined #openstack-swift17:58
*** dmorita has quit IRC17:59
*** dmorita has joined #openstack-swift18:00
*** zhill has joined #openstack-swift18:02
*** klrmn has joined #openstack-swift18:04
*** zhill has quit IRC18:04
*** zhill has joined #openstack-swift18:05
*** garthb has quit IRC18:07
pchngclayg: I submitted a small PR to vagrant SAIO to fix the proxy server config to reflect the trivial_keymaster -> keymaster rename: https://github.com/swiftstack/vagrant-swift-all-in-one/pull/3218:09
claygpchng: thanks for the heads up!18:12
*** jordanP has quit IRC18:15
*** rohit_ has joined #openstack-swift18:17
pdardeauany ideas on the highest num. partitions that folks use?18:24
*** silor has joined #openstack-swift18:25
claygpdardeau: I'd *freak* out if anyone used anything higher than 24 - 22 is a much more likely upper bound18:26
zaitcevI think RAX was 18 all around until the last couple of years.18:26
claygwhat time is gerrit supposed to come back?18:27
ahalezaitcev: rax was 2018:27
ahalewell, is i suppose18:27
zaitcevahale: I see, mine is from defaults in gholt's old master class18:27
pdardeauclayg: thx!18:28
*** daemontool__ has joined #openstack-swift18:28
pdardeauclayg: part of why i asked is i got segfaults doing rebalance on 2418:30
claygadd more memory to your vm18:33
clayg;)18:33
*** briancurtin has quit IRC18:33
*** briancurtin has joined #openstack-swift18:34
*** aix has quit IRC18:35
pdardeauclayg: i took the other option (doc, it hurts when i do this. don't do that)18:35
*** trifon has joined #openstack-swift18:43
*** zhill has quit IRC18:45
notmynamehello clayg18:46
claygnotmyname: hey!18:47
notmynameclayg: are you coming in tot he office today, or just digging out from home?18:47
claygi'm wfh today - i'm planning on coming in tomorrow18:47
notmynamegot it18:47
notmynameclayg: (very) short summary of swift is ring stuff landed. testr functests landed. need to get eyes on copy middleware and concurrent reads18:48
clayggot most of the junk email out of the way - only ~100 I want to read, but a lot of the ones I'm interested in point to gerrit18:48
notmynameoh, we did a swiftclient release18:48
claygnotmyname: sounds awesome!18:48
claygnotmyname: great work to all!18:48
notmynameyeah, lots of progress18:48
*** david-lyle_ has joined #openstack-swift18:49
*** david-lyle has quit IRC18:50
notmynameclayg: I made a small PR to vSAIO to add some words around using it as a test target18:50
*** david-lyle_ is now known as david-lyle18:54
*** lpabon has quit IRC18:58
*** eranrom has joined #openstack-swift19:00
onovynotmyname, new swiftclient release? 2.7.0?19:02
*** daemontool__ has quit IRC19:09
notmynameonovy: yeah. happened last week19:09
onovythat's all, already building deb package :)19:09
onovybut good news everyone: swauth 1.1 is out19:10
*** daemontool has joined #openstack-swift19:10
notmynamecool19:10
onovy /all/old/ :]19:10
*** zhill has joined #openstack-swift19:15
*** david-lyle has quit IRC19:17
*** david-lyle has joined #openstack-swift19:20
*** silor has quit IRC19:21
*** silor has joined #openstack-swift19:23
*** dmorita has quit IRC19:24
*** david-lyle_ has joined #openstack-swift19:24
*** david-lyle has quit IRC19:24
*** acoles is now known as acoles_19:25
*** dmorita has joined #openstack-swift19:27
pdardeaunotmyname: any suggestions on how to test ring with large number of devices?19:28
pdardeau60000-70000 ish19:28
notmynamepdardeau: you don't need any servers to do that. you just need some ram and CPU. probably would be good to talk to torgomatic (not online?) about his ring fuzzing tests19:29
onovyis swiftclient 2.7.0 for liberty or mitaka release?19:29
timburkemitaka19:29
onovyok, thanks19:29
notmynamethere will be others in the mitaka release too19:30
onovyof course :)19:30
pdardeaunotmyname: ok, i'll do that19:30
*** david-lyle_ has quit IRC19:30
pdardeaunotmyname: so how would i experience the 65535 device limit without using any servers?19:31
notmynamepdardeau: the limits are only in the builder and in the builder operations. you don't need to actually deploy a ring to test that it has more than 2**32 drives. likewise you don't even need to have servers or drives to add it to a ring19:32
zaitcevimport swift.common.ring as rm;  r = rm.Ring(something=65636)19:32
zaitcevI guess19:32
notmynameyeah :-)19:32
zaitcevand use ulimit(1) to see how much RAM is used19:32
pdardeaunotmyname: got it! i'm able to add 70000 devices right now to the ring19:32
notmynamewith current master code?19:32
pdardeaunotmyname: yes19:33
notmynameso write_ring and make sure you can actually get all of them19:33
notmynamethe dev_id serialization has the 2**16 limit19:33
pdardeaubut not the way zaitcev has shown above19:33
pdardeaui have rb.add_dev in loop19:34
zaitcevpdardeau: but that's the test after you added and rebalanced it all. I thought you wanted to see what server does without actual server.19:34
pdardeauzaitcev: i see, thx for the explanation19:35
*** trifon has quit IRC19:35
zaitcevpdardeau: Like notmyname says, builder is not run on a server node or in a context of a server process. Builders are run on a management workstation. So that is what you're accelerating. After that, the *.ring.gz goes to servers and then it becomes interesting how much RAM that big a ring uses up.19:35
notmynamezaitcev: it's not just accelerating. also removing any device count limits (really that's the main thing)19:36
notmynamepdardeau: I'm looking for that part of the code now19:36
pdardeauzaitcev: i knew the part of the builder being offline cli mechanism, but thanks for mentioning19:38
notmynamepdardeau: I think you'll be mucking around in the swift/common/ring/ring.py:RingData class19:41
notmynameespecially the [de]serialize_v1() methods19:41
notmynameie you'll make a *_v2 for those19:41
notmynameassuming there is backwards incompatibility19:41
*** zhill has quit IRC19:41
ahalepdardeau: dunno if you found it yet, but doing a rebalance should make it error ?19:44
notmynamepdardeau: also, I'd suggest only using a part power of 17 to make rebalances faster19:46
notmynameie dont use 24 or something ;-)19:47
*** david-lyle has joined #openstack-swift19:48
pdardeauahale: thanks! that's what i was looking for, but not asking correctly (the symptom)19:49
*** trifon has joined #openstack-swift19:50
*** daemontool has quit IRC19:50
ahaleno worries - ( like https://gist.github.com/racker-andrew-hale/acac080b71df32674a7b )19:50
*** bsdkurt has quit IRC19:53
*** lpabon has joined #openstack-swift19:55
ZyricMorning19:59
notmynameZyric: hello19:59
notmynamepdardeau: basically, it comes down to the array.array structure of replica2part2dev_id20:05
*** lyrrad has quit IRC20:05
*** zhill has joined #openstack-swift20:05
*** lyrrad has joined #openstack-swift20:06
pdardeaunotmyname: got it, thx20:06
*** diogogmt has joined #openstack-swift20:10
*** zhill has quit IRC20:10
*** lyrrad has quit IRC20:15
*** minwoob has quit IRC20:17
*** lyrrad has joined #openstack-swift20:19
*** trifon has quit IRC20:20
*** silor has quit IRC20:27
*** timur___ has joined #openstack-swift20:28
*** tsg has quit IRC20:33
*** changbl has joined #openstack-swift20:37
*** tsg has joined #openstack-swift20:41
*** trifon has joined #openstack-swift20:45
*** tsg has quit IRC20:47
onovywhat the f*ck with gerrit... :)20:53
notmynameonovy: upgrade. should be turned on at the top of the hour. see the topic message20:53
onovynotmyname, it's up 'n' running20:53
notmynameoh cool20:53
onovyand f*cked :)20:53
onovynew fancy look, etc.20:54
notmynameoh. yeah. I've played with that before. I'm told it's better this time20:54
*** daddyjoseph97 has joined #openstack-swift20:55
*** ho has joined #openstack-swift20:55
notmynameonovy: oh wow. yeah this is really different20:55
onovyreally :)20:55
mattoliverauMorning20:55
hrouonovy, wow yea, and no way to revert back this time I think :)20:56
onovyhrou, can't found anything20:57
mattoliverauI'm too scared to go look20:57
onovyargh command for check-out is there. tiny download button on top right20:57
*** tsg has joined #openstack-swift20:57
hogood morning!20:57
*** CaioBrentano has quit IRC20:57
notmynameit will take a while to get used to20:57
onovyyep20:57
notmynameI hope it's better than the last time I tried the new look20:58
*** acoles_ is now known as acoles20:58
mattoliverau+120:58
kota_good morning20:59
notmynamemeeting time. looks like the previous one is finishing up21:00
acoleskota_: mattoliverau : good morning21:00
notmynamein #openstack-meeting21:00
*** bjkeller has joined #openstack-swift21:01
*** ChanServ changes topic to "Review Dashboard: https://goo.gl/gSuaEG | Get an IRC bouncer at http://znc.in | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/ | DocImpact changes http://lists.openstack.org/pipermail/openstack-dev/2015-November/080294.html"21:02
-openstackstatus- NOTICE: Gerrit has been upgraded to 2.11. Please report any issues in #openstack-infra as soon as possible.21:02
*** trifon has quit IRC21:03
openstackgerritOndřej Nový proposed openstack/swift: Keystone middleware deprecated option is_admin removed  https://review.openstack.org/25815821:05
pchngMmm new gerrit. Interesting UI changes21:05
*** ober37 has joined #openstack-swift21:06
*** bjkeller has quit IRC21:07
*** bjkeller has joined #openstack-swift21:08
*** daemontool has joined #openstack-swift21:08
*** changbl has quit IRC21:11
onovypchng, __interesting__ :)21:11
pchngIt's... good I guess? Looks like the UI is a little "flatter", some stuff moved around. Not sure what is new. I guess the +/- add/remove indicators is now a green/red bar?21:13
mattoliverauit's going to take me some getting used to, I hope I don't abandon anyones changes by default :P21:14
*** zhill has joined #openstack-swift21:15
*** trifon has joined #openstack-swift21:18
*** esker has quit IRC21:20
*** daemontool_ has joined #openstack-swift21:22
*** daemontool__ has joined #openstack-swift21:22
*** daemontool has quit IRC21:23
*** trifon has quit IRC21:23
*** daemontool_ has quit IRC21:26
*** dmorita has quit IRC21:27
*** rcernin has joined #openstack-swift21:28
*** dmorita has joined #openstack-swift21:28
acolesmattoliverau: you can switch off the coloured bars in settings, and siwtch on patch number in listings which is a + imho21:29
*** ober37 has quit IRC21:32
*** ober37 has joined #openstack-swift21:34
*** ober37 has quit IRC21:35
*** ober37 has joined #openstack-swift21:35
acolesnotmyname: do you see all patches in dashboard belong to Anonymous Coward? or is just me?21:48
notmynamejust you21:48
acoleshmmm21:49
*** daemontool_ has joined #openstack-swift21:52
openstackgerritMerged openstack/swift: remove unused parameters in the method  https://review.openstack.org/24340121:52
*** daemontool__ has quit IRC21:56
*** daemontool_ has quit IRC21:56
tdasilvaacoles: I saw Anonymous Coward when I was not signed in...21:58
*** ober37 has quit IRC21:59
acolestdasilva: oic it shows me signed in but maybe i need to refresh. thanks22:00
tdasilvaacoles: nvm, it's not that, I saw it once and now can't replicate it22:00
notmynamejrichli: torgomatic seems to be not online (and is out sick). clayg is in, but is catching up22:01
jrichlinotmyname: thanks.22:01
onovytrying again here: if someone can get me opinion for patch 251151 ; just say localtime or gmt, that's all :)22:02
patchbotonovy: https://review.openstack.org/#/c/251151/ - Show local time in swift-recon in replication part22:02
acolesnotmyname: so, the issue with ranged gets is, afaik a multipart response has unpredicatble number of parts, each of which, for us has an encrypted content-type header which is longer than the plaintext. So I'm not sure how we calculate the content-length sent to the client without reading the entire body.22:02
acolesnotmyname: but I'm wondering if EC didn't have to deal with similar22:03
*** daemontool has joined #openstack-swift22:04
acolestdasilva: i signed out/in and all is good now22:04
acolestdasilva: "good" being relative :)22:04
blmartincschwede: onovy: I updated patch 257577 so if you get bored today22:05
patchbotblmartin: https://review.openstack.org/#/c/257577/ - Print min_part_hours lockout time remaining22:05
claygoh - change spark lines in gerrit now!22:05
*** bjkeller has quit IRC22:07
onovyblmartin, i will look22:08
*** ianbrown has quit IRC22:08
blmartinonovy: much appreciated22:08
onovyand peterlisak will look too, right peter? :)22:08
*** ianbrown has joined #openstack-swift22:08
acolesclayg: looks like the 'f' kbd shortcut has gone from gerrit :( that was SO useful22:10
mattoliveraucool in gerrit settings you can add dash boards (you could probaby do this before but I never looked). So now the swift review dashboard in under the 'My' menu in gerrit.22:11
peterlisakonovy, blmartin , yes, sure but tomorrow ... time to go off :)22:11
acolesand we can edit code in gerrit as well as commit messages. scary,22:11
acolesmattoliverau: wasn't the swift dashboard already your browser home page? ;)22:13
claygacoles: everything gerrit did that didn't annoy you is gone - all the things you hate are worse - and there's new things you didn't even think of that they invinted just to make you less productive22:13
mattoliverauacoles: yeah.. of course, bookmarked, but now I can add it as apart of my gerrit login22:13
blmartinclayg: but it looks soo much flashier though doesn't it! think of how stylish your monitor will look at conferences with gerrit open now22:15
acolesclayg: precisely22:15
claygblmartin: fair point - sounds like a net win to me22:15
*** diazjf has quit IRC22:15
acolesmattoliverau: oic and you can make any dashboard be the default page by moving it to top22:15
*** lpabon has quit IRC22:15
*** diazjf has joined #openstack-swift22:16
acolesargh, and 'r' is no longer 'leave review' but does something else!!22:16
mattoliveraulol22:16
mattoliverauR is reload change22:17
acolesdon't they realise that some of us are too old to learn new ways!22:17
onovyhey, why is "reply" button at top of page, not near to comments? :/22:17
acoleslittle r is mark this file as reviewd22:17
mattoliverauonovy: so you can scroll back up and hit reply :P22:17
acolesits to give you time to reconsider22:18
mattoliverauacoles: oh yes, toggle the review flag22:18
onovyblmartin, just one small thing, but maybe i'm missing something. just read it, will try to test it tomorrow, thank you22:18
onovymattoliverau, :) that's it!22:18
mattoliverauonovy: 'a' is review and publish comments, I hope not all at once.22:19
acolesI do like that I can have the review number as a column in the dashboard. I must focus on that positive.22:19
onovymattoliverau, thanks, opens comment dialog22:19
onovyrows in tables are higher. this is not good, because i still have 100+ outgoing reviews and that list is really long :/22:20
onovyand i'm not sure if that "size" column is good. nobody will open "long" reviews now :)22:22
acolesonovy: agree. you can turn it off in settings.22:22
blmartinonovy: likely it is I who is missing something, I look forward to the comments22:23
onovyacoles, no i like it. i will do review for "short" commits only :)22:23
mattoliveraulol22:24
*** zhill has quit IRC22:25
acolesi wonder, if we have one enormous review will all the others go green?22:25
blmartinwell, breaking a patch into 30 parts probably isn't that bad I guess22:25
onovywith gerrit deps it's cool. Commit message: First part of my change, does nothing. Commit message2: Second part of my change, still does nothing....22:26
*** blmartin has quit IRC22:27
acolesok i am done whinging, good night.22:27
clayg'night22:28
timburkeacoles: i'm guessing no, but only because the opposite doesn't seem to be true: https://review.openstack.org/#/q/project:openstack/swift3+is:open+-label:Workflow-1+-label:Verified-122:28
onovyi'm going too. good night22:28
brianclineg'night22:29
*** diazjf has quit IRC22:31
*** acoles is now known as acoles_22:32
*** asettle is now known as asettle-sick22:39
*** daemontool has quit IRC22:40
*** zhill has joined #openstack-swift22:45
*** diazjf has joined #openstack-swift22:47
mattoliverauacoles_, onovy night22:47
*** diazjf has quit IRC22:48
openstackgerritTimur Alperovich proposed openstack/swift: Fix IPv6 handling in MemcacheConnPool.  https://review.openstack.org/25870422:55
*** daemontool has joined #openstack-swift22:55
kota_clayg: !!!22:59
*** tsg has quit IRC23:00
*** tsg has joined #openstack-swift23:04
*** km has joined #openstack-swift23:06
*** alejandrito has quit IRC23:08
*** daemontool has quit IRC23:09
*** tsg has quit IRC23:13
*** zhill has quit IRC23:15
*** rohit_ has quit IRC23:18
timur___quit23:23
*** timur___ has quit IRC23:23
*** dustins is now known as dustins|out23:30
*** kei_yama has joined #openstack-swift23:31
*** zhill has joined #openstack-swift23:40
*** changbl has joined #openstack-swift23:48
openstackgerritTim Burke proposed openstack/swift: Expose token expiration time in tempauth auth response  https://review.openstack.org/25872023:56
*** chlong has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!