Saturday, 2015-12-19

notmynamethat's a good question00:00
notmynamemaybe00:00
notmyname:-)00:00
*** haomaiwang has quit IRC00:01
notmynameone thought is to keep it really simple and just have account names00:01
notmynameanother idea is to collect more of the account stat info00:01
*** haomaiwang has joined #openstack-swift00:01
*** trifon has quit IRC00:02
notmynamehowever, with more info, it will lead to more complexity later. eg if you store the bytes count you find for each DB, then people will want that to be reported. and then you'll have to figure out how to reconcile it when you get different answers from different replicas in the cluster00:02
notmynameand by you I mean all of us00:02
openstackgerritvenkatamahesh proposed openstack/swift: OpenStack typo  https://review.openstack.org/24360800:03
notmynameon the other hand, although it will lead to more complexity, the other account info is useful. ie if you don't store/report it, then the first thing someone will do when they get a list of accounts is HEAD each of them to get that info00:03
then3rdI have my logging back to normal now, there are a couple errors I've been seeing.. are these normal, or should I figure out what's causing it? "invalid user token - deferring reject downstream" and "Client disconnected on read (txn: ...)"00:03
openstackgerritEran Rom proposed openstack/swift: Container-Sync to iterate only over synced containers  https://review.openstack.org/20580300:03
notmynameZyric: I think you should leave it to a simple list of accounts for now00:03
notmynameZyric: and absolutely the first line of the file you are producing should be the version of the file. this is a version 1. that way we can change it later if needed00:04
Zyricnotmyname: Cool I'll keep it simple for now. I suppose what is collected depends on need/intent. If you want to collect historical data or just know what exists right now.00:05
notmynamethen3rd: the first is normal-ish. it's keystone saying that it thinks the token probably isn't good, but the ACLs that are loaded later might in fact make the request ok (eg a publicly available object). it's just normal keystone logging00:05
notmynamethen3rd: the second is also normal, or at least expected. clients sometimes disconnect before finishing the whole download00:05
notmynamethen3rd: however, if you're seeing a lot of those, it might point to some other problem00:06
notmynameZyric: right. and we can add complexity later. it's really hard to remove it once it's added00:06
Zyricnotmyname: I should probably make a formal file somewhere that holds this account data then? Is there a good place for this in the system or should I direct it to temp?00:07
*** eranrom has quit IRC00:07
notmynameZyric: I'm imagining that you will write a file on disk (not in /tmp) that will ahve the info. then part 2 is getting swift-recon to read it00:08
*** zhill has quit IRC00:10
Zyricnotmyname: Me too. Swift-recon was my next investigation :) So far I quite like the watcher hooks, is it okay to progress in this direction and assume that this is a good direction for swift or should it be discussed a bit more? I know torgomatic's patch is still in review.00:11
then3rdnotmyname: Thanks, that's a simpler answer than I found anywhere. Are there some good first steps or tests that come to mind for the second one? I can't say I've ever seen a failed GET.. since it's streaming video I 'm not supprised to see so many disconnets.00:15
notmynameoh, yeah for streaming video that would be very common, I'd think00:16
notmynamereally it's just a process of monitoring it and seeing how the rate changes over time00:16
then3rdAnd judging by my logs/graphs, the errors come in bursts of no more than 5/minute, if that.00:16
notmynamedisconnects are logged as a 499 response code in the logs00:17
notmynamethen3rd: with anything in swift, the first thing is to grab that transaction id in the logs (the txn) and grep all the logs in the cluster for that. any log line associated with that request will have the same transaction id00:17
*** haomaiw__ has joined #openstack-swift00:29
*** haomaiwang has quit IRC00:32
*** gyee has joined #openstack-swift00:32
*** Jeffrey4l has quit IRC00:40
*** gyee has quit IRC00:45
*** haomaiw__ has quit IRC01:01
*** 5EXAAJA62 has joined #openstack-swift01:01
*** badari has joined #openstack-swift01:07
clayg^ mattoliverau that reminds me concurrent gets needs to watch out for lp bug #140930201:16
openstackLaunchpad bug 1409302 in OpenStack Object Storage (swift) "Use of GreenAsyncPile can lose txn_id logging" [Medium,Confirmed] https://launchpad.net/bugs/140930201:16
*** badari has quit IRC01:17
*** badari has joined #openstack-swift01:54
*** 5EXAAJA62 has quit IRC02:01
*** haomaiwang has joined #openstack-swift02:01
*** badari_ has joined #openstack-swift02:08
*** badari has quit IRC02:12
*** klrmn has quit IRC02:12
*** changbl has quit IRC02:36
*** haomaiwang has quit IRC02:43
*** janonymous_ has joined #openstack-swift02:49
*** changbl has joined #openstack-swift02:55
*** dmorita has quit IRC02:58
*** klrmn has joined #openstack-swift03:13
mattoliverauclayg: good call, I'll make a note to check we are :)03:15
*** haomaiwang has joined #openstack-swift03:16
*** janonymous_ has quit IRC03:22
*** klrmn has quit IRC03:49
*** haomaiwang has quit IRC04:01
*** haomaiwang has joined #openstack-swift04:01
*** nadeem has joined #openstack-swift04:02
*** breitz has quit IRC04:11
*** proteusguy_ has quit IRC04:22
*** proteusguy_ has joined #openstack-swift04:34
*** klrmn has joined #openstack-swift04:46
*** haomaiwang has quit IRC05:01
*** haomaiwang has joined #openstack-swift05:01
*** mac_ified has quit IRC05:08
*** nadeem has quit IRC05:13
*** changbl has quit IRC05:27
*** klrmn has quit IRC05:29
*** thumpba has quit IRC05:38
*** changbl has joined #openstack-swift05:40
*** haomaiwang has quit IRC06:01
*** haomaiwang has joined #openstack-swift06:01
*** trifon has joined #openstack-swift06:16
*** trifon has quit IRC06:40
*** badari_ has quit IRC06:51
*** trifon has joined #openstack-swift06:56
*** haomaiwang has quit IRC07:01
*** haomaiwang has joined #openstack-swift07:01
*** zaitcev has quit IRC07:09
*** haomaiwang has quit IRC08:01
*** haomaiwang has joined #openstack-swift08:01
*** haomaiwang has quit IRC08:20
*** links has joined #openstack-swift08:50
*** haomaiwa_ has joined #openstack-swift08:51
*** trifon has quit IRC08:55
*** haomaiwa_ has quit IRC09:01
*** haomaiwang has joined #openstack-swift09:01
*** jordanP has joined #openstack-swift09:06
*** trifon has joined #openstack-swift09:08
*** janonymous_ has joined #openstack-swift09:25
*** jordanP has quit IRC09:30
*** links has quit IRC09:47
*** haomaiwang has quit IRC10:01
*** haomaiwang has joined #openstack-swift10:01
*** haomaiwang has quit IRC11:01
*** haomaiwang has joined #openstack-swift11:01
*** jordanP has joined #openstack-swift11:52
*** haomaiwang has quit IRC12:01
*** haomaiwang has joined #openstack-swift12:01
*** jordanP has quit IRC12:42
*** haomaiwang has quit IRC13:01
*** haomaiwang has joined #openstack-swift13:01
*** haomaiwang has quit IRC14:01
*** haomaiwang has joined #openstack-swift14:01
*** cebruns has quit IRC14:20
*** cebruns has joined #openstack-swift14:25
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873614:39
*** changbl has quit IRC14:40
*** changbl has joined #openstack-swift14:59
*** haomaiwang has quit IRC15:01
*** haomaiwang has joined #openstack-swift15:01
*** tsg_ has joined #openstack-swift15:07
*** tsg has quit IRC15:07
*** tsg has joined #openstack-swift15:10
*** tsg_ has quit IRC15:10
*** changbl has quit IRC15:31
*** diogogmt has quit IRC15:33
*** dslevin has joined #openstack-swift15:35
*** dslevin has quit IRC15:41
*** changbl has joined #openstack-swift15:44
*** jordanP has joined #openstack-swift15:52
*** mac_ified has joined #openstack-swift15:52
*** haomaiwang has quit IRC16:01
*** haomaiwang has joined #openstack-swift16:01
*** jordanP has quit IRC16:14
*** janonymous_ has quit IRC16:42
*** silor has joined #openstack-swift16:51
*** silor1 has joined #openstack-swift16:57
*** silor has quit IRC16:58
*** silor1 is now known as silor16:58
*** haomaiwang has quit IRC17:01
*** haomaiwang has joined #openstack-swift17:01
*** janonymous_ has joined #openstack-swift17:01
*** silor1 has joined #openstack-swift17:09
*** silor has quit IRC17:11
*** silor1 is now known as silor17:11
openstackgerritMerged openstack/swift: Make ECDiskFile report all fragments found on disk  https://review.openstack.org/23112117:15
*** badari has joined #openstack-swift17:21
*** badari_ has joined #openstack-swift17:30
*** whydidyoustealmy has joined #openstack-swift17:31
*** patchbot has quit IRC17:31
*** patchbot` has joined #openstack-swift17:32
*** scott_ has joined #openstack-swift17:32
*** torgomatic has quit IRC17:32
*** Guest30753 has quit IRC17:32
*** cschwede has quit IRC17:32
*** sc68cal has quit IRC17:32
*** cschwede has joined #openstack-swift17:32
*** cschwede has quit IRC17:32
*** cschwede has joined #openstack-swift17:32
*** torgomatic_ has joined #openstack-swift17:32
*** scott_ is now known as Guest8725817:32
*** bwall has quit IRC17:32
*** patchbot` is now known as patchbot17:32
*** bwall has joined #openstack-swift17:32
*** bwall has joined #openstack-swift17:32
*** badari has quit IRC17:33
*** barra204 has quit IRC17:34
*** sc68cal has joined #openstack-swift17:34
*** badari_ has quit IRC17:47
*** nadeem has joined #openstack-swift17:55
*** haomaiwang has quit IRC18:01
*** haomaiwang has joined #openstack-swift18:01
*** eranrom has joined #openstack-swift18:28
*** nadeem has quit IRC18:45
*** klrmn has joined #openstack-swift18:50
*** haomaiwang has quit IRC19:01
*** haomaiwang has joined #openstack-swift19:01
*** tsg has quit IRC19:26
*** mac_ified has quit IRC19:56
*** jordanP has joined #openstack-swift19:58
*** haomaiwang has quit IRC20:01
*** haomaiwang has joined #openstack-swift20:01
*** jordanP has quit IRC20:03
*** nadeem has joined #openstack-swift20:06
*** nadeem has quit IRC20:46
*** haomaiwang has quit IRC21:01
*** haomaiwang has joined #openstack-swift21:01
*** esker has joined #openstack-swift21:12
*** trifon has quit IRC21:14
*** nadeem has joined #openstack-swift21:39
*** esker has quit IRC21:50
*** silor has quit IRC21:54
*** esker has joined #openstack-swift21:57
*** haomaiwang has quit IRC22:01
*** haomaiwang has joined #openstack-swift22:01
jrichlinotmyname: great infogram!22:03
*** nadeem has quit IRC22:20
*** eranrom has quit IRC22:36
*** nadeem has joined #openstack-swift22:54
*** haomaiwang has quit IRC23:01
*** haomaiwang has joined #openstack-swift23:01
*** david-lyle has quit IRC23:44

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!