Wednesday, 2014-06-18

torgomaticmattoliverau: perfect! looks like "-label:Code-Review>=-2,self" is exactly what I'm after :)00:02
mattoliverautorgomatic: cool :)00:13
*** kevinc_ has quit IRC00:20
*** dmorita has joined #openstack-swift00:29
*** mwstorer has quit IRC00:33
*** shri has quit IRC00:34
*** kopparam has joined #openstack-swift00:40
*** kopparam has quit IRC00:45
openstackgerritOpenStack Proposal Bot proposed a change to openstack/python-swiftclient: Updated from global requirements  https://review.openstack.org/8925000:47
openstackgerritOpenStack Proposal Bot proposed a change to openstack/swift: Updated from global requirements  https://review.openstack.org/8873600:48
clayglol guess what binds more closely than python's ternary conditional operator -> https://github.com/openstack/swift/blob/master/swift/common/db.py#L47200:50
torgomaticclayg: conn.execute("outgoing") you say?00:52
claygtorgomatic: sqlite seems to think it's a syntax error - but that codes always been like that - so it's obviously wrong00:54
torgomaticclayg: maybe that nobody calls get_syncs() with incoming=False? at least not on my current branch00:55
claygtorgomatic: yeah I think everyone uses get_sync - it's vistigial code - not surprising it doesn't work00:57
torgomaticclayg: I see a whopping one caller, and that's in db_replicator.py's _usync_db method, and it doesn't pass incoming=False00:58
torgomaticbut that's pretty funny that this code is obviously broken, but nobody ever calls it that way so we get by without crashing00:59
mattoliveraulol, wow00:59
*** scohen has joined #openstack-swift01:14
*** scohen has quit IRC01:14
*** matsuhashi has joined #openstack-swift01:17
clayglandmines are awesome01:21
claygzaitcev: portante: so did you guys have some ideas on merge_timestamps?  I've been beefing up test coverage assuming some "what about if you did like this" comment was in the works?01:26
zaitcevclayg: working on it right this minute. my plan is 1. split merge_timestamps into account (old) container (new, with is_deleted), 2. pass arguments such as conn all through, 2a. watch out for deadlocks01:27
claygzaitcev: interesting... and in building up the diff you feel it'd look better now than later?01:29
zaitcevclayg: Not sure about looks as such, after all I introduce some trampolining (versus core duplication). But at least it won't add as many DB open-closes and closes races Peter mentioned.01:30
zaitcevclayg: And yes, I know what Knuth said.01:30
zaitcevsorry it took so long01:31
*** nosnos has joined #openstack-swift01:31
claygthe race is the one where two different calls to merge_timestamps are bothing changing the status of the database one from deleted->recreated and the other recreated->deleted and neither end up changing status_changed_at - cause I acctually think we'd be ok to live with that for a little while.01:35
zaitcevokay01:36
claygwhere "live with that for a little while" basically means I don't think it really matters in that case because you have two other db's that already disagree and now one that agrees with at least one of them - regardless the storage_policy_index will eventually resolve to one of them01:37
claygthe multiple cursor checkout thing is a little tricker, the old code was a bit more methodical about stitching together the queries - but like in the is_deleted, get_info case it worked out to less actual connections; OTOH I don't really think those were all that expensive to begin with...01:40
zaitcevI see...01:40
claygzaitcev: what *did* Knuth say about merge_timestamps anyway?01:41
zaitcevHe said that the premature optimization is the root of all evil.01:41
zaitcevWhich is often taken to mean that it's acceptable to make code as crappy as it happens and "tune" it later, which does not necessarily follow.01:42
claygzaitcev: heh01:43
claygI think i made a comment I before that I was attempting to optomize that routine for readability and correctness - but I may have failed to deliver either :D01:44
*** openfly has joined #openstack-swift01:52
*** scohen has joined #openstack-swift02:14
*** scohen has quit IRC02:16
*** nsquare has quit IRC02:19
*** asdfsf has joined #openstack-swift02:21
*** annegentle has joined #openstack-swift02:21
*** matsuhashi has quit IRC02:29
*** matsuhashi has joined #openstack-swift02:32
*** haomai___ has quit IRC02:36
*** zhiyan_ is now known as zhiyan02:44
*** haomaiwa_ has joined #openstack-swift02:45
portanteclayg, zaitcev: I don't think that the issue is about readability or optimization, it is about making decisions based on stale knowledge and executing against it02:59
*** gyee has quit IRC02:59
portanteIn this case, we have the facilities to make the decision within one transaction that avoids such cases03:00
*** matsuhashi has quit IRC03:00
portanteI think we should consider merging this as one transaction03:00
*** kajinamit has joined #openstack-swift03:02
zaitcevThey are hard to pack into this because get_info() flushes the pending.03:04
zaitcevThe old is_deleted did it too.03:05
zaitcevI am looking at open-coding is_deleted() now.03:05
zaitcevThe after-SP is_deleted() is even thicker than before. It's huge.03:05
portanteopen-coding?03:06
zaitcevrow = conn.execute('''SELECT put_timestamp, delete_timestamp, object_count FROM container_stat''').fetchone()03:08
zaitcevit's ugly too03:09
*** scohen has joined #openstack-swift03:15
*** scohen has quit IRC03:15
*** nosnos has quit IRC03:27
openstackgerritZhang Hua proposed a change to openstack/swift: Add distributed tracing capablities in logging.  https://review.openstack.org/9367703:29
claygportante: I think we are considering it, and it ends up being a tradeoff on readability/maintainability03:46
claygzaitcev: right now I'm hung up reasoning about the difference in how accounts/containers reason about the is_deleted (status vs <db_contains>_count)03:47
zaitcevclayg: I'm about to e-mail you a diff03:47
claygzaitcev: oh good, everything i'm trying ends up either being a bunch of dupliation or spaghetti03:49
zaitcevI'm using the gmail address, is that okay03:50
claygzaitcev: yeah whatever, just get up the diff man!  ;)03:51
zaitcevI feel like this was the "mountain bourne a mouse" kind of thing03:52
zaitcevand an ugly mouse at that03:52
zaitcevbut hey, it's a transaction like portante observed03:52
*** matsuhashi has joined #openstack-swift03:53
claygzaitcev: k i see what you did, i think the idea of passing around the conn is mostly reasonable03:53
zaitcevMy own motivation was that remember how we discussed that wee shuld not have any "callbacks" anymore (unless passed as arg, like a real callback). It was like the #1 outcome of hackathon to me03:54
zaitcevNext week we're at work and your adding these is_deleted() calls into derived class from base class' merge_timestamps03:54
zaitcevLike duuuuuude, what did we have the freaking hackathon for03:54
zaitcev(well it was fun)03:55
claygzaitcev: I guess common.db shouldn't really have a merge_timestamps method, and the account should just say "with self.get() as conn: self._merge_timestamps(conn, *some_args)" ?03:55
zaitcevSo I felt challenged to rearrange the code so it follows our agreement03:55
zaitcevunless it's impossible, of course - no need to be too dogmatic03:56
claygzaitcev: well as I said, the sp work sorta prompted me to realize how much the existing layout needed some dogmatism03:57
zaitcevclayg: oh crap you're right03:57
zaitcevclayg: no, wait, scared myself too early. I took yor comment to mean that I forgot the with conn:03:58
claygzaitcev: no i think what you have is correct, I was just looking the duplication in the middle there between container.merge_timestamps and common._merge_timestamps04:00
zaitcevclayg: I figured you needed to think in broader categories because SP was so big. So, you declare: status is changed when the deleted status is changed. Then you reason about it, without regard to how much crap is necessary to run is_deleted().04:00
zaitcevclayg: yeah, sadly it's duplicated04:00
*** charz has joined #openstack-swift04:01
zaitcevclayg: give me a sec to re-upload04:03
zaitcevactually, re-running unit tests and pep8 just in case04:04
*** nosnos has joined #openstack-swift04:07
zaitcevhttp://www.zaitcev.us/things/swift-sp-merge_del-2.diff04:08
*** matsuhashi has quit IRC04:12
*** matsuhashi has joined #openstack-swift04:12
claygzaitcev: yeah that's basically where I ended up04:13
zaitcevoh04:13
claygzaitcev: but looking at who else uses update_status_changed_at... It's all the same mess in _update_or_create all over again04:13
claygalso, it's maybe annoying that accounts don't get their status_changed_at "fixed"04:15
zaitcevI meant to ask you about it04:15
*** scohen has joined #openstack-swift04:15
zaitcevDoes it do anything at all?04:15
zaitcevI see why it's needed in containers04:16
*** scohen has quit IRC04:16
zaitcevBut I may be missing something crucial.04:16
claygi'm pretty luke warm at this effort, esspecially this late in the cycle (having been burnt on the per policy tempdir stuff) - i agree "mountain out of molehill"04:16
claygzaitcev: no it's not critical for accounts (currently)04:16
zaitcevYou're just tired. But we can fix it up later, yes.04:17
*** matsuhashi has quit IRC04:17
claygzaitcev: but when we were doing the original prototyping of storage polices it ended up being a huge hassel that there was this field called "status_changed_at" that didn't acctually include the time when the damn db changed status - and we're just leaving that for the next guy on accounts04:17
*** haomaiwa_ has quit IRC04:17
*** haomaiwang has joined #openstack-swift04:18
*** annegentle has quit IRC04:18
zaitcevI am going to drink some tea and sleep04:18
claygwell, g'night04:18
*** matsuhashi has joined #openstack-swift04:21
*** zaitcev has quit IRC04:27
*** haomai___ has joined #openstack-swift04:27
*** ppai has joined #openstack-swift04:29
claygwell shit, if we're worried that we get the wrong behavior taking action on stale info - I wonder why we shoudln't have to call commit_puts?04:29
*** haomaiwang has quit IRC04:31
*** matsuhashi has quit IRC04:36
*** matsuhashi has joined #openstack-swift04:36
*** matsuhashi has quit IRC04:39
*** matsuhashi has joined #openstack-swift04:39
*** ppai has quit IRC04:40
*** psharma has joined #openstack-swift04:45
*** scohen has joined #openstack-swift04:49
*** scohen has quit IRC04:53
*** matsuhashi has quit IRC05:11
*** matsuhashi has joined #openstack-swift05:11
*** annegentle has joined #openstack-swift05:14
*** annegentle is now known as Guest2592605:14
*** matsuhashi has quit IRC05:16
*** nsquare has joined #openstack-swift05:16
*** matsuhashi has joined #openstack-swift05:17
*** Guest25926 has quit IRC05:19
*** Midnightmyth has joined #openstack-swift05:22
*** matsuhashi has quit IRC05:24
*** matsuhashi has joined #openstack-swift05:24
*** kopparam has joined #openstack-swift05:24
*** scohen has joined #openstack-swift05:29
*** ppai has joined #openstack-swift05:33
*** annegentle has joined #openstack-swift05:44
*** annegentle is now known as Guest6658105:45
*** Guest66581 has quit IRC05:51
*** kopparam has quit IRC06:02
*** kopparam has joined #openstack-swift06:05
*** psharma has quit IRC06:19
*** openstackgerrit has quit IRC06:19
*** omame has quit IRC06:19
*** foexle has joined #openstack-swift06:23
*** psharma has joined #openstack-swift06:23
*** openstackgerrit has joined #openstack-swift06:23
*** omame has joined #openstack-swift06:23
*** kopparam is now known as vm06:24
*** vm is now known as kopparam06:25
*** scohen has quit IRC06:28
*** scohen has joined #openstack-swift06:28
*** miqui has quit IRC06:36
*** annegentle has joined #openstack-swift06:46
*** annegentle is now known as Guest4419106:46
mattoliverauI'm calling it a day, night all06:46
*** Guest44191 has quit IRC06:50
*** kopparam has quit IRC06:51
*** kopparam has joined #openstack-swift06:52
*** kopparam has quit IRC07:02
*** kopparam has joined #openstack-swift07:03
*** kopparam has quit IRC07:07
*** matsuhashi has quit IRC07:08
*** matsuhas_ has joined #openstack-swift07:08
*** asdfsf has quit IRC07:14
*** chandan_kumar has quit IRC07:17
*** nshaikh has joined #openstack-swift07:22
*** charz has quit IRC07:25
*** toolslive has joined #openstack-swift07:31
*** kopparam has joined #openstack-swift07:44
*** annegentle has joined #openstack-swift07:46
*** annegentle is now known as Guest936907:47
*** nacim has joined #openstack-swift07:50
*** jamiehannaford has joined #openstack-swift07:50
*** Guest9369 has quit IRC07:51
*** nacim has quit IRC07:55
openstackgerritClay Gerrard proposed a change to openstack/swift: Fixes probe tests with non-zero default storage policy  https://review.openstack.org/9635307:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add two vector timestamps  https://review.openstack.org/9931507:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy Support  https://review.openstack.org/9602707:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy Documentation  https://review.openstack.org/9602607:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy support to Containers  https://review.openstack.org/9602907:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Update FakeRing and FakeLogger  https://review.openstack.org/9602807:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy support to Object Server  https://review.openstack.org/9603007:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Fix object-expirer for missing objects  https://review.openstack.org/9851107:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add functional tests for Storage Policy  https://review.openstack.org/9604207:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Update bin scripts to be storage policy aware  https://review.openstack.org/9604307:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Merge container storage_policy_index  https://review.openstack.org/9604007:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy Support to Accounts  https://review.openstack.org/9604107:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy Support to list_endpoints  https://review.openstack.org/9604607:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy Support to Container Sync  https://review.openstack.org/9604707:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Enqueue misplaced objects during container replication  https://review.openstack.org/9604407:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add LRUCache to common.utils  https://review.openstack.org/9604507:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy Support to ssync  https://review.openstack.org/9603407:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Put X-Backend-Timestamp in object 404 responses  https://review.openstack.org/9603507:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy Support to the Auditor  https://review.openstack.org/9603207:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add storage policy support for the Replicator  https://review.openstack.org/9603307:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add container-reconciler daemon  https://review.openstack.org/9603807:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add reconciler probetest outline  https://review.openstack.org/9603907:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Extend interface on InternalClient  https://review.openstack.org/9603607:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Extend direct_client  https://review.openstack.org/9603707:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy support to Object Updates  https://review.openstack.org/9997907:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy support to the Account Reaper  https://review.openstack.org/9604907:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Add Storage Policy Support to Recon Middleware  https://review.openstack.org/9604807:56
openstackgerritClay Gerrard proposed a change to openstack/swift: Refactoring storage policies merge_timestamps  https://review.openstack.org/10080707:56
*** nsquare has quit IRC07:58
*** nacim has joined #openstack-swift08:00
claygi think there's still something to do on the timestamp offset width, the list endpoints v2 response, and replacing the header constants with literals08:00
claygbut the probetest should be fixed, and I took a stab at something for merge_timestamps08:01
*** kopparam has quit IRC08:05
*** kopparam has joined #openstack-swift08:05
*** kopparam has quit IRC08:07
*** kopparam has joined #openstack-swift08:08
*** Midnightmyth has quit IRC08:10
ahaleI can't wait to deploy all this to a prod cluster, see what happens - so much change!08:11
*** kopparam has quit IRC08:12
omameI'm already preparing the popcorn08:12
*** ppai has quit IRC08:13
*** mmcardle has joined #openstack-swift08:22
*** toolslive has quit IRC08:22
*** haomai___ has quit IRC08:28
*** kopparam has joined #openstack-swift08:30
*** ppai has joined #openstack-swift08:30
claygnom nom nom08:34
*** toolslive has joined #openstack-swift08:35
*** Longgeek_ has joined #openstack-swift08:53
*** mkollaro has joined #openstack-swift08:53
*** matsuhas_ has quit IRC09:01
*** mmcardle has quit IRC09:02
*** mmcardle has joined #openstack-swift09:03
*** matsuhashi has joined #openstack-swift09:09
*** mkerrin has quit IRC09:22
*** mkollaro has quit IRC09:33
*** matsuhashi has quit IRC09:38
*** dANOKELOFF has joined #openstack-swift09:39
dANOKELOFFHi guys, I try to implement tempurl on swift but when i execute my curl command i have this error : 401 Unauthorized: Temp URL invalid , anyone can help me ?09:40
*** mkollaro has joined #openstack-swift09:41
*** toolslive has quit IRC09:48
*** zhiyan is now known as zhiyan_09:51
*** dANOKELOFF has quit IRC09:53
*** matsuhashi has joined #openstack-swift10:01
*** dANOKELO_ has joined #openstack-swift10:03
*** matsuhashi has quit IRC10:06
*** Kbee has joined #openstack-swift10:08
*** matsuhashi has joined #openstack-swift10:08
*** scohen has quit IRC10:09
*** scohen has joined #openstack-swift10:14
*** kajinamit has quit IRC10:14
*** nacim has quit IRC10:16
*** matsuhashi has quit IRC10:17
*** matsuhashi has joined #openstack-swift10:18
*** kopparam has quit IRC10:20
*** kopparam has joined #openstack-swift10:21
*** Trixboxer has joined #openstack-swift10:22
*** dmorita has quit IRC10:25
*** kopparam has quit IRC10:25
*** mkerrin has joined #openstack-swift10:27
*** kopparam has joined #openstack-swift10:37
*** Kbee has quit IRC10:38
*** mmcardle has quit IRC10:41
*** mmcardle has joined #openstack-swift10:44
*** mmcardle has quit IRC10:44
*** mmcardle has joined #openstack-swift10:47
*** Longgeek_ has quit IRC10:58
*** tdasilva_ has joined #openstack-swift10:59
*** tdasilva has joined #openstack-swift11:00
*** tdasilva has quit IRC11:00
*** tdasilva_ is now known as tdasilva11:00
rieglflohi guys, while paginating through objects within an container, is there any way to check whether i'm on the first or the last page? to check if object count is smaller than the limit, is just not valid. and when using the end_marker to page back the same issue comes up for the first page ... i never know when i have reached the end. any recommendations?11:30
*** d0ugal has quit IRC11:33
*** d0ugal has joined #openstack-swift11:34
*** mmcardle has quit IRC11:40
*** kopparam has quit IRC11:43
*** kopparam has joined #openstack-swift11:43
*** ppai has quit IRC11:46
*** kopparam has quit IRC11:47
*** matsuhashi has quit IRC11:48
*** kopparam has joined #openstack-swift11:49
*** matsuhashi has joined #openstack-swift11:51
*** matsuhashi has quit IRC11:56
*** matsuhashi has joined #openstack-swift11:56
*** scohen has quit IRC11:56
dANOKELO_can we remove manually the expire url in swift ?11:58
*** scohen has joined #openstack-swift11:59
*** ppai has joined #openstack-swift11:59
*** matsuhashi has quit IRC12:01
*** nacim has joined #openstack-swift12:03
*** ppai has quit IRC12:13
*** tdasilva_ has joined #openstack-swift12:21
*** tdasilva has quit IRC12:22
*** annegent_ has joined #openstack-swift12:23
*** annegent_ is now known as annegentle_12:24
*** kopparam has quit IRC12:25
*** kopparam has joined #openstack-swift12:25
*** mmcardle has joined #openstack-swift12:28
*** kopparam has quit IRC12:30
ahalechmouel: do you have to wear a fedora now not your cool homebrew baseball cap ?12:32
omameha12:33
ahaledANOKELO_: no you can't expire it earlier without changing the key, you can create a new different one though if its because you had problems with the 401'ing one earlier12:33
ahaleseems a common request recently though12:34
dANOKELO_ahale: we cant delete the tempurl ?12:34
ahalenope, theres nothing to delete really12:34
chmouelahale: i guess i need to work on my redhat pose yeah ;)12:34
ahaleyou can invalidate the hmac thing but thats about it12:35
ahalechmouel: heheh12:35
*** tdasilva_ is now known as tdasilva12:37
dANOKELO_ahale: ok thank's12:43
rieglfloa config setting for the tempurl plugin with max expire seconds would be enough ... e.g. all links with an expiry greater than time() + max_expire_seconds are invalid12:45
rieglfloso the max age of tempurls can be configured on the server side ...12:46
*** annegentle_ has quit IRC12:53
*** sombrafam has joined #openstack-swift12:56
openstackgerritChangBo Guo(gcb) proposed a change to openstack/swift: Add common units constants  https://review.openstack.org/7649013:01
*** annegent_ has joined #openstack-swift13:03
*** lpabon has joined #openstack-swift13:05
peluse_clayg:  good stuff man - do you some some sort of shift differential for working graveyard?  :)13:06
*** annegent_ has quit IRC13:13
*** psharma_ has joined #openstack-swift13:14
*** navid__ has joined #openstack-swift13:14
*** nshaikh has quit IRC13:17
*** psharma has quit IRC13:17
*** psharma__ has joined #openstack-swift13:19
*** nshaikh has joined #openstack-swift13:19
*** navid__ has quit IRC13:22
*** psharma_ has quit IRC13:22
*** psharma__ has quit IRC13:22
*** Kbee has joined #openstack-swift13:28
peluse_portante:  you around?13:31
*** annegent_ has joined #openstack-swift13:35
*** annegent_ is now known as annegentle_13:36
creihtwow.. congrats to the enovance folkd13:37
creihtfolks13:37
*** annegentle_ has quit IRC13:40
*** zz_wasmum is now known as wasmum13:41
*** annegent_ has joined #openstack-swift13:41
wasmumGreat Morning, all.13:42
wasmumsay, anyone have experience with swift3 from stackforge, using it with keystone?  I would like to have some conversation later, if possible.13:43
*** miqui has joined #openstack-swift13:46
*** Longgeek_ has joined #openstack-swift13:57
*** dmsimard_away is now known as dmsimard14:02
*** nosnos has quit IRC14:03
*** asdfsf has joined #openstack-swift14:03
*** pberis has joined #openstack-swift14:04
*** mmcardle has quit IRC14:05
*** mmcardle has joined #openstack-swift14:07
*** annegent_ has quit IRC14:09
*** annegent_ has joined #openstack-swift14:10
peluse_portante:  saw in process note on gerrit... OK, cool.  I was fixing things up as well but will stop.  I'm sure you noticed that reload_storage_policies() is using local swift.conf and not the one defined in __init__.py, that's what I was just looking at...14:15
portantethat is what I was fixing and have that working now14:18
peluse_portante:  awesome14:19
*** esmute has quit IRC14:22
*** haomaiwang has joined #openstack-swift14:23
*** esmute has joined #openstack-swift14:25
*** mwstorer has joined #openstack-swift14:29
*** asdfsf has quit IRC14:30
*** kopparam has joined #openstack-swift14:31
*** Longgeek_ has quit IRC14:43
*** scohen has quit IRC14:43
*** annegent_ has quit IRC14:57
thurloatwasmum I'm going to be looking at deploying swift3 on keystone in the coming months, care to document your woes for my future self? :)14:59
*** foexle has quit IRC15:00
*** mmcardle has quit IRC15:00
*** tdasilva has quit IRC15:03
*** kevinc_ has joined #openstack-swift15:03
*** jergerber has joined #openstack-swift15:04
*** annegent_ has joined #openstack-swift15:04
*** tdasilva has joined #openstack-swift15:05
*** kopparam has quit IRC15:05
*** kopparam has joined #openstack-swift15:06
*** zaitcev has joined #openstack-swift15:06
*** ChanServ sets mode: +v zaitcev15:06
*** mmcardle has joined #openstack-swift15:08
notmynamegood morning, world15:09
*** byeager has joined #openstack-swift15:10
*** kopparam has quit IRC15:11
dmsimardnotmyname: good morning :)15:11
notmynameanother day, another Red Hat acquisition :-)15:11
zaitcevnow what15:13
zaitcevNot SwiftStack I'm sure15:13
notmynamezaitcev: you bought eNovance15:15
notmynamechmouel: cschwede: does this mean drinks are on you in Paris? ;-)15:15
*** nshaikh has quit IRC15:17
peluse_that sounds logical to me15:18
*** Kbee has quit IRC15:19
creihthaha15:21
notmynamethe SP patches have been under review for quite some time now. soft freeze of master as well. I know that there are improvements being made and bugs found, but we need some resolution soon15:21
zaitcevIt was my sense too. But I'm behind on reviewing and testing it and get focused on small things.15:22
portantenotmyname: true, but what is that saying? no cheese before wine? ;)15:24
notmynameportante: oh I can whine quite a bit15:24
peluse_notmyname:  is it reasonable to vote on merge to master and address any other lingering items prior to cutting RC (so as to end the freeze and keep the ball moving forward)15:24
portantewhether it lands today or another day, still going to be reviewing that patch chain as we go15:25
chmouelnotmyname: yeah there would be plenty of champagne tonight :)15:25
notmynamepeluse_: I think that is a reasonable suggestion (it does make progress), but also working towards an actual release is important15:26
peluse_yup, agree just seems like we're a little "stuck" and can make faster progress on the smaller things by getting this monster chain mgmt out of the way15:27
notmynamethe current plan is to merge (and see what else needs to land, but that's mostly handled already), cut an RC and have that for a couple of weeks, then release15:27
notmynamepeluse_: so you're saying just get the mechanics of the merge out of the way (ie propose the merge commit and then just vote on that)? rather than focus on the patch chain itself more?15:28
notmynameonce SP is merged, "normal" life should return where we address new patches and fix discovered bugs15:29
peluse_notmyname:  yes, I'm saying if there are now showstoppers lets merge to master and relieve clayg of managing the chain.  This also allows other to directly fix SP related items (right now nobody pushes on one of claygs patches because it messes up the process)15:29
notmynamepeluse_: s/now/no/15:30
notmynamepeluse_: yes, that makes a lot of sense to me15:30
peluse_yes :)15:30
notmynamepeluse_: why is it always the single most important word in the sentence that gets the typo? ;-)15:30
peluse_notmyname:  lots of recent examples of tweaks that could have been by several other folks that we saddled clayg with because of the chain... even one current (in process func tests)15:31
portantecan I be a small thorn, and just request a little time to update my patch chain locally in my SAIOs and verify that latest?15:31
peluse_notmyname:  yeah, my typos always totally change the meaning of what I'm trying to say :)15:31
notmynameportante: of course. I don't think anyone is doing anything in the next couple of hours :-)15:32
notmynameportante:  suspect clayg might still be asleep ;-)15:32
peluse_notmyname:  I think clayg just went to bed a few hrs ago :)15:32
peluse_ha!15:32
*** kevinc_ has quit IRC15:33
portantenotmyname: great, thanks15:33
notmynamepeluse_: I'm told the avotons are back online now (thanks mlanner!!). I'm going to check on them today15:34
goodesgood morning all15:34
notmynamepeluse_: portante: I myself am going to go get ready now. I'll be back online later15:35
*** jyoti_ranjan has joined #openstack-swift15:35
portantek15:35
peluse_notmyname:  cool, curious to see the root cause of those dead mobos... not super surprising though, that's a brand new board and you got one of the first batches15:36
jyoti_ranjanI have one question related to Swift deployment.15:36
jyoti_ranjanCan we have more than one Swift deployed in a given cloud? If yes how endpoint will be registered to cloud?15:36
* peluse_ also needs to get organized and head into the office...15:36
*** scohen has joined #openstack-swift15:37
*** kevinc_ has joined #openstack-swift15:39
*** pberis has quit IRC15:46
jyoti_ranjanI know that people do not use this use case typically (i.e. two Swift). I am asking because there is need to have two Swift setup: one for internal cloud usage and another for end consumer..15:47
*** mmcardle has quit IRC15:47
jyoti_ranjanCertianly alterante option is to have on Swift and use different account for differnt purpose. But wanted to know whether it is technially feasable to deploy two Swift in a given cloud or not.15:48
*** mmcardle has joined #openstack-swift15:49
*** pberis has joined #openstack-swift15:49
*** bsdkurt has joined #openstack-swift15:50
*** Kbee has joined #openstack-swift15:50
*** haomaiwang has quit IRC15:51
*** haomaiwang has joined #openstack-swift15:52
creihtjyoti_ranjan: yeah you can do that, you just have two different end points15:52
zaitcevA typical client won't be able to select them unless region is specified15:55
zaitcevI don't know how clients default when facing two.15:56
jyoti_ranjanzaitcev: can I use same keystone for both deployment?15:56
zaitcevjyoti_ranjan: Yes, by specifying a region.15:56
jyoti_ranjanAlso, I do see that there is a field call region as an attribute in keystone which we use at the time of registering end point.15:57
*** Kbee has quit IRC15:57
jyoti_ranjanlet us say, I have two endpoitns: region 1 for internal, region 2 for external.15:57
creihtcorrect, each one has to be a separate region15:58
jyoti_ranjanCan cinder running in region 1 can use end point of Swift running in region 2/15:58
creihtthat I do not know15:58
jyoti_ranjanI want o use for volume backup15:58
jyoti_ranjanCinder just needs Swift backpoint but not sure across region it will work or not.15:59
*** rieglflo has quit IRC15:59
*** tdasilva has quit IRC15:59
creihtif it is just an endpoint in cinder, then you should be fine15:59
*** gyee has joined #openstack-swift16:00
jyoti_ranjanGreat, it helps. We need something like backup_swift_url=http://localhost:8080/v1/AUTH_ and its associated user name and password etc16:04
*** dANOKELO_ has quit IRC16:07
*** haomaiwang has quit IRC16:08
*** pberis has quit IRC16:11
*** pberis has joined #openstack-swift16:12
openstackgerritDonagh McCabe proposed a change to openstack/swift: Add swift-checker utility for use by monitoring tools  https://review.openstack.org/9996116:12
*** chandan_kumar has joined #openstack-swift16:18
mlannerpeluse_: i think we may actually have another (5th) bad mobo. i'll have to do some surgery later today. <- notmyname16:18
*** elambert has joined #openstack-swift16:21
wasmumthurloat: re: documentation, I'll try and share the discovery with you16:21
thurloatwasmum thanks a bunch.16:22
*** cHilDPROdigY1337 is now known as nottrobin16:26
gholtnotmyname: Do we have a copyright policy yet? https://review.openstack.org/#/c/76490/16:31
*** ezml has quit IRC16:32
notmynamegholt: actually, I think the copyright line(s) in that patch are doing the right thing. openstack doesn't require copyright assignment (and in fact the foundation doesn't want assignment)16:36
*** nacim has quit IRC16:36
notmynamegholt: so aside from the weirdness of "powers of 2 are now (c) IBM" the copyright in that patch is good16:37
notmynamegholt: note that I think the patch is pretty frivolous, though, and my comments on the copyright are independent of if that patch should land (I'd tend to "no")16:37
notmynameI'll leave a comment16:38
gholtOkay cool, so my listing of everybody that touched a file for copyright purposes is still fine? :)16:39
notmynamegholt: maybe. in that case you're getting into the concepts of "work for hire" etc. which may in fact be different for different companies and different countries :-)16:40
notmynamewhich is why I mostly stay away from all of it :-)16:40
gholtHehe, yeah. It's still the root swift/__init__.py suddenly being all "HP!" that riles me. I don't mind folks adding their stamps, but kind of pretending you did it all is a bit off.16:41
*** haomaiwa_ has joined #openstack-swift16:43
gholtIf I submitted a pull request to somebody's project and plopped my copyright on their root file, giving no other credit, I would expect to be soundly told "no".16:45
notmynamegholt: even if you replaced the entire contents of the file, except for the license header?16:45
notmyname(not disagreeing, just continuing the conversation)16:46
gholtYes, unless that was the entire project in that one file.16:46
gholtFor an analogy, it'd be like a book cover artist only having his name on the cover.16:47
*** haomaiwa_ has quit IRC16:48
notmynameonce the SP patches land, I think there will be some churn to that file. If you remember, it turns out that our versioning stuff isn't "right", so we'll be adjusting anyway. And if we end up touching __init__, then we can fix it :-)16:49
torgomaticdoes this mean I need to figure out if I put my name or my company's name in copyright statements now? :|16:49
notmynametorgomatic: only after you figure out how much you care16:50
gholtHeheh, you could just use my "who worked on this?" script, modified to use a .companymap file. ;)16:51
gholtOr .copyrightmap file I guess, heh16:51
torgomaticnotmyname: I'm not particularly upset about it... maybe 7 or 8 angststroms?16:51
*** byeager has quit IRC16:52
gholtOn a scale of 1-10! omg!16:53
gholtOh, maybe angststroms are logarithmic...16:54
torgomatic:)16:55
*** Midnightmyth has joined #openstack-swift16:57
gholtbtw, I think I'm about done as much as I can be with the SP reviews. My brain has gotten to the point where it's not really seeing new things anymore. Though I did realize after a bit that I wasn't even working off the tail anymore. :/16:57
torgomaticgholt: got any feedback to post?16:58
gholtJust what I have already16:58
torgomaticcool16:59
peluse_gholt: anything specific for discussion here?16:59
gholtJust be sure, 96026 is the tail now, right? :)16:59
peluse_yes17:00
torgomaticI'm sure there'll be a trail of bugs to fix post-merge, but that's true of just about any patch on just about any project17:00
peluse_especially of this magnitude.. totally agree17:00
peluse_there'd better only be 1 or 2 more tmp-N dir bugs though :)17:01
gholtI do like that when I diff against master, about 2/3 are tests.17:02
notmyname:-)17:02
peluse_gholt:  and that was no accident!17:02
notmynameya, the fact that such a huge feature is actually increasing test coverage is nice :-)17:02
notmynamegood job peluse_ and clayg and torgomatic and portante!17:03
*** shri has joined #openstack-swift17:03
torgomaticso it looks like there's a little more pending work on the 2-vector timestamp thing, but the rest seems okay...?17:04
* torgomatic is just trying to get a sense of known problems17:04
*** jyoti_ranjan has quit IRC17:06
peluse_torgomatic:  you mean extending the offset width?17:06
torgomaticpeluse_: yeah, that17:06
torgomaticI think clayg's on top of it17:07
peluse_torgomatic:  OK, I didn't have a strong opinion either way but if extending then agree now would be the time17:07
torgomaticpeluse_: I'm with you there; I don't really care how wide it is, but adding more later will be hard17:07
* torgomatic goes quiet for the walk from train to office17:08
gholtMostly I think we just need to be sure no one ever uses the second stamp to change actual content. I don't think POST or delete could use them, for instance. It could cause sync problems external to the cluster.17:10
gholtOh, I guess I should leave that as a comment on the commit message.17:13
claygmorning all, just read back through my scroll buffer17:13
notmynamegood morning clayg17:13
claygnot sure how to say redhat totally baffles me without sounding like I don't think the enovance team is compleatly awesome - which of course they are - but i really don't get redhat...17:14
peluse_morning17:14
peluse_watch out or they may "get" you :)17:15
claygi think i said before i went to bed - there's like three things outstanding...17:15
*** cutforth has joined #openstack-swift17:15
claygtimestamp offset width, the list endpoints v2 response, and replacing the header constants with literals17:15
peluse_clayg:  I think the last 2 can wait til SP lands on master no?17:16
clayggholt: i think we need the offset on delete, don't know if I could/should exclude it on post exactly...17:16
claygoh idk, maybe - yeah acctually the header replace would be a lot easier in one patch than 2717:17
claygthe list endpoints thing should be easy enough - it's just more to review - so it's really mostly about how effectively I can write obviously correct code :\17:17
*** annegent_ has quit IRC17:17
claygdid anyone look at the merge_timestamps change?  Is that better - after it was all said in done I think I liked it better - but maybe it was just different17:18
clayganyway, i need to get out of the house, i think the timestamp offset width fix is just changing a constant - so i'll do that and update the effected tests.17:18
*** nsquare has joined #openstack-swift17:20
*** byeager has joined #openstack-swift17:22
*** tdasilva has joined #openstack-swift17:24
claygacoles: I wonder if we shouldn't do <post_timestamp>_<data_file_timestamp> or something so that the last_modified in the container listing looks more like when the post happened than when the datafile got uploaded?  dunno.17:25
claygbbiab17:28
*** byeager has quit IRC17:31
acolesclayg: hmm, but data file timestamp must always take precedence when sorting. could maybe add the offset (which would be post_time - put_time) to 'normal' part when reporting.17:39
acolesclayg: commute time for me.17:42
*** haomaiwa_ has joined #openstack-swift17:44
*** haomaiwa_ has quit IRC17:49
*** tong_ has joined #openstack-swift17:52
*** mmcardle has quit IRC17:52
*** Trixboxer has quit IRC17:55
*** sombrafam has quit IRC17:59
*** scohen has quit IRC18:00
*** scohen has joined #openstack-swift18:01
*** kevinc_ has quit IRC18:04
*** scohen has quit IRC18:05
torgomaticfor those trying to figure out which SP patches they've left reviews on and which they haven't, particularly after a rebase, here's a Gerrit search I find useful:18:16
torgomaticstatus:open branch:master topic:bp/storage-policies -label:Code-Review>=-2,self    <--- patches without a vote from me on their latest change set18:16
*** byeager has joined #openstack-swift18:17
torgomaticnow that we're on Gerrit 2.newer-than-before, we have the "self" user, which refers to the currently-logged-in user, so there's no need to go s/torgomatic/$me/ on the query18:17
*** byeager has quit IRC18:22
portantetorgomatic: thanks18:22
torgomaticportante: np18:22
torgomaticif you want to see things you *have* left reviews on, just s/-label/label/18:22
peluse_torgomatic:  hmm, it didn't like 'self' for me, I had to use 'peluse'18:26
torgomaticpeluse_: are you logged in?18:26
peluse_oh jeeze...18:26
peluse_thanks :)18:26
torgomaticpeluse_: np :)18:28
*** gvernik has joined #openstack-swift18:30
*** wasmum is now known as zz_wasmum18:36
*** zz_wasmum is now known as wasmum18:38
claygacoles: yeah i was thinking something like that - it maybe makes sense to continue to update the user facing last_modified on post like post-as-copy does today18:40
notmynameswift team meeting in 20 minutes in #openstack-meeting18:41
*** byeager has joined #openstack-swift18:42
claygso my train was full of kids on there way to some summer camp field trip - didn't make any progress on the timestamp change - also sorta waiting to see if gholt could elaborate on comment on the two vector timestamps18:42
peluse_clayg:  sounds like you missed a good opportunity to collaborate with some fresh thinkers...18:46
portantenotmyname: gonna be late to the swift team meeting, bunch of stuff goin' on here today18:48
creihtnotmyname: I'll be there for a bit, but will hae to jet a bit early for the meeting18:48
creihtfor a meeting18:48
notmynameok. mostly I want to raise the question I asked this morning. how do we continue to make progress and actually deliver software? clayg has mentioned just a few outstanding things, and others have said that at least some can be done post-merge18:50
*** annegent_ has joined #openstack-swift18:50
creihtI'm kind of in the let's get it merged and iterate group18:50
zaitcevyeah18:51
notmynamecreiht: I think most of us are trending that way :-)18:51
*** annegent_ has quit IRC18:53
*** annegent_ has joined #openstack-swift18:54
torgomaticyep, same here18:58
*** ozialien has joined #openstack-swift18:59
zaitcevIt's like Obamacare: you've got to pass it to find what's in it. And if you like your current containers, you can keep them!19:00
notmynameand it's noon19:01
peluse_zaitcev:  yikes!19:01
notmynamezaitcev: well except that you've been able to see what's in it every step of the way ;-)19:01
zaitcevnotmyname: I feel like I'm too stupid to deduce the implications, so it's no use. I have to rely on Sam and Clay in the end19:03
notmynameclayg: we're talking storage policies in #openstack-meeting19:05
*** lpabon has quit IRC19:06
*** openstackgerrit_ has joined #openstack-swift19:06
*** openstackgerrit has quit IRC19:06
*** openstackgerrit_ is now known as openstackgerrit19:08
*** annegent_ has quit IRC19:13
*** ozialien has quit IRC19:18
*** jamiehannaford has quit IRC19:26
*** wasmum is now known as zz_wasmum19:27
*** kevinc_ has joined #openstack-swift19:35
*** jamiehannaford has joined #openstack-swift19:39
notmynamegvernik: I don't think we'll be able to cover apache spark stuff in today's meeting. I'm interested though, and maybe later this afternoon (if you are traveling) or tomorrow am (so TZs work) I'd love to hear about it19:45
gverniksure, let's cover it next time.19:46
*** zz_wasmum is now known as wasmum19:48
*** gvernik has quit IRC19:51
peluse_notmyname:  missed the tail end of the mtg, are we merging today?19:52
mattoliverauWell I'm heading back to bed, see everyone in a few hours :)19:54
notmynamepeluse_: clayg will widen the timestamps and push the last copy of the chain this afternoon or tonight. from there, I'll work with mordred and get the force push of the chain upstream, and propose the merge commit to master. at that point I'll ask for a "Review of record" for people on the merge commit and then we'll merge that.19:55
peluse_awesome!19:55
mordrednotmyname: I did a test of the process on sandbox and it works just fine, btw19:56
notmynamemordred: cool!19:56
notmynameand anything else will be done on master after SP lands. and at the SP merge time, we'll tag the RC and ask the community to test it. then after 2 weeks, we'll cut the final (having backported any major fixes as necessary to the RC)19:56
mordrednotmyname: otoh - the gate is _WAY_ more healthy today than it was last week19:56
mordrednotmyname: so there is the chance that merging normally could Just Work™19:57
mordred:)19:57
notmynamemordred: yes, but still the idea of having just one commit for SP included is much better19:57
mordredsure. just saying19:57
*** kevinc_ has quit IRC19:58
*** tdasilva has quit IRC19:58
*** scohen has joined #openstack-swift20:04
shriHey guys, question about containers… can a container be deleted when it is not empty?20:04
*** scohen has quit IRC20:05
*** mwstorer has quit IRC20:11
glangeno20:17
peluse_shri: a container can't be deleted if it has stuff in it (but I think the swift CLI for example has a flag that will let you do this because it deletes contents behind the scenes for you)20:17
shriI see that the swift command line client deletes all blobs and then deletes the container20:18
shriI am looking at a swift instance where the /srv/node/r0/objects directory has several files in it but there are no containers. Is that because of a bug?20:18
peluse_shri:  I think there is a bug there that one of my colleagues ran into as well... one sec20:19
peluse_wait, thought you were asking something different20:19
peluse_shri: there could still be stuff in there, depends on what you are seeing as to whether or not it would be expected or not20:20
*** annegent_ has joined #openstack-swift20:21
peluse_.ts files for example could still be there for a while after you've deleted everything20:21
shrioh.. thats right.. Let me check20:22
*** foexle has joined #openstack-swift20:23
*** annegent_ has quit IRC20:26
*** byeager has quit IRC20:28
*** byeager has joined #openstack-swift20:29
*** annegent_ has joined #openstack-swift20:33
claygzaitcev: portante: can you guys put a high pass filter on https://review.openstack.org/#/c/100807/ - if i'm going to be doing more work there tonight i'd like to know it asap20:34
portanteclayg: looking at it now20:41
claygportante: cool, thanks!20:46
*** nsquare has quit IRC20:51
*** byeager has quit IRC20:53
*** byeager has joined #openstack-swift20:53
*** miqui has quit IRC20:54
*** mmcardle has joined #openstack-swift21:00
*** kevinc_ has joined #openstack-swift21:03
*** tongli has joined #openstack-swift21:12
*** tongli has quit IRC21:13
*** byeager has quit IRC21:16
*** byeager has joined #openstack-swift21:17
*** byeager has quit IRC21:19
*** wasmum is now known as zz_wasmum21:20
*** tong_ has quit IRC21:20
claygok, i think i got the 16 digit timestamp fix in21:21
*** byeager has joined #openstack-swift21:22
*** zz_wasmum is now known as wasmum21:23
*** jergerber has quit IRC21:26
*** fifieldt has quit IRC21:27
*** wasmum is now known as zz_wasmum21:36
*** jamiehannaford has quit IRC21:37
*** fifieldt has joined #openstack-swift21:40
*** byeager has quit IRC21:40
*** Midnightmyth has quit IRC21:43
*** annegent_ has quit IRC21:44
notmynameFYI there is a low-impact security bug that has been reported for Swift. I've just marked the bug public, and I'm about to push the patch for master and the backport to Icehouse21:46
notmynamehttps://bugs.launchpad.net/swift/+bug/132741421:46
*** byeager has joined #openstack-swift21:47
openstackgerritJohn Dickinson proposed a change to openstack/swift: properly quote www-authenticate header value  https://review.openstack.org/10103121:47
*** mmcardle has quit IRC21:50
*** mmcardle has joined #openstack-swift21:50
*** mmcardle has quit IRC21:51
*** byeager has quit IRC22:03
*** byeager has joined #openstack-swift22:07
portanteclayg, notmyname: fwiw, I have run the sp patch chain through it paces in my two SAIOs and saw no problems.  Not that that will really indicate much, ...22:07
notmynamethanks22:07
*** scohen has joined #openstack-swift22:07
portantein the coming days after it lands, I'll post the patches that get the in-process functional tests working with sp, and addition of the in-memory diskfile backend as a storage policy22:07
*** scohen has quit IRC22:07
peluse_portante:  cool22:08
portanteI'd like to see the core devs contribute to ramping up the coverage of some of the base infrastructure code once sp lands on master (like for common/db.py, etc.) as a way to help vet this work further22:10
*** dmsimard is now known as dmsimard_away22:10
notmynameand it looks like a pep8 error on that security patch. *sigh*22:14
*** byeager_ has joined #openstack-swift22:16
claygheheheheh -> http://openstackreactions.enovance.com/2014/03/paeeeep8/22:16
*** byeager has quit IRC22:21
torgomaticyep, still good :)22:25
notmynamemtreinish: sorry about that. the patch had a flak8 bug. fixed now. https://review.openstack.org/#/c/101032/22:29
mtreinishnotmyname: heh, ok. +2'd again22:36
notmynamethnaks22:36
*** byeager_ has quit IRC22:49
mattoliveraumorning all22:55
notmynamemattoliverau: hello, again.23:02
notmynamemattoliverau: you got up for the meeting! wasn't it exciting?23:02
*** ozialien has joined #openstack-swift23:03
mattoliveraunotmyname: lol, yeah I made it, tired but made it :) The meeting was fun, exciting as descisions were made and looks like things are moving forward (SP wise). And it isn't as early as I thought it would be, it was only 5am (on the east coast of australia).. I still went back to bed after, but still not as early :P23:12
notmynameA group of swifts are collectively known as a "screaming frenzy"23:16
mattoliveraunotmyname: ROFL!!! that is a t-shirt for the next sumit :)23:21
* mattoliverau going to find some breakfast, brb.23:22
*** mordred has quit IRC23:24
*** mordred has joined #openstack-swift23:27
*** zz_wasmum is now known as wasmum23:29
*** mordred has quit IRC23:30
*** mordred has joined #openstack-swift23:30
openstackgerritA change was merged to openstack/swift: properly quote www-authenticate header value  https://review.openstack.org/10103123:38
*** asdfsf has joined #openstack-swift23:48
claygso in extending the tests for the merge_timestmaps change to cover the account and container brokers I rememberd that the create_account_stat_table code wasn't updated to set status_changed_at to put_timestamp during initialize - i'm inclined to fix that now23:58
*** sungju has joined #openstack-swift23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!