Thursday, 2017-03-23

openstackgerritTim Burke proposed openstack/swift master: Version DLOs, just like every other type of object  https://review.openstack.org/44614200:12
openstackgerritTim Burke proposed openstack/swift master: Factor out a bunch of common testing setup  https://review.openstack.org/44618500:22
openstackgerritTim Burke proposed openstack/swift master: Factor out a bunch of common testing setup  https://review.openstack.org/44618500:30
*** tonanhngo has quit IRC00:33
*** chsc has quit IRC00:41
*** tonanhngo has joined #openstack-swift00:41
*** zhurong has joined #openstack-swift00:43
*** tonanhngo has quit IRC00:45
*** tonanhngo has joined #openstack-swift00:47
*** tonanhngo has quit IRC00:52
*** lifeless_ is now known as lifeless01:00
*** _JZ_ has quit IRC01:00
*** vinsh has joined #openstack-swift01:06
*** vinsh has quit IRC01:09
*** klrmn has quit IRC01:13
*** m_kazuhiro has joined #openstack-swift01:18
m_kazuhirogood morning01:19
*** zhurong has quit IRC01:23
*** NM has joined #openstack-swift01:24
*** NM has quit IRC01:40
*** zhurong has joined #openstack-swift01:42
*** ntata has joined #openstack-swift01:48
*** mmotiani has joined #openstack-swift01:51
*** vinsh has joined #openstack-swift01:57
*** sgundur- has joined #openstack-swift01:58
tone_zGood morning01:59
*** vinsh has quit IRC02:01
*** _JZ_ has joined #openstack-swift02:04
mattoliveraum_kazuhiro, tone_z: morning02:06
tone_z:)02:07
*** _JZ_ has quit IRC02:08
m_kazuhiromattoliverau, tone_z: morning!02:14
mattoliverauHere is one backport to fix one itermittant stable/newton swift error. https://review.openstack.org/44885402:15
*** sams-gleb has joined #openstack-swift02:16
mattoliverauAlthough it isn't the same failure that timburke was seeing. infact none of them are the same.. they are all different errors. But will conintue running tests to see if I can hit another one.02:16
mattoliveraualso, gotta love git bisect to find these patches to back port :)02:17
*** dmorita has quit IRC02:17
*** sams-gleb has quit IRC02:20
m_kazuhiromattoliverau: Is the url link [1] in your commit message correct? The link shows me "File Not Found".02:21
tone_zme too02:23
mattoliveraum_kazuhiro: ahh, well it's a back port, so it's cschwede's old commit message, and we only keep logs for so long, so there probably gone now.02:23
mattoliverauProblem is, I hit this locally, so don't have a link, but maybe I'll update the commit message to point to the error in question.02:23
mattoliveraum_kazuhiro, tone_z: thanks both of you :)02:24
tone_z:-)02:25
openstackgerritjunboli proposed openstack/swift master: keep consistent naming convention of swift and urls  https://review.openstack.org/44738702:28
openstackgerritjunboli proposed openstack/python-swiftclient master: Be consistent in using example.com instead of identity.api.rackspacecloud.com  https://review.openstack.org/44885602:31
mattoliverauk, changed it to a paste.o.o link with the failed test error you receive02:32
tone_zIt is OK now. ;)02:36
m_kazuhirome too :)02:36
mattoliverauta :)02:41
*** dmorita has joined #openstack-swift02:42
*** dmorita has quit IRC02:43
*** dmorita_ has joined #openstack-swift02:43
*** dmorita_ has quit IRC02:44
*** dmorita has joined #openstack-swift02:44
*** dmorita_ has joined #openstack-swift02:46
*** dmorita has quit IRC02:49
*** dmorita_ has quit IRC02:50
*** dmorita has joined #openstack-swift02:55
*** dmorita_ has joined #openstack-swift02:56
*** dmorita_ has quit IRC02:57
*** dmorita_ has joined #openstack-swift02:57
*** dmorita has quit IRC02:59
*** dmorita_ has quit IRC03:02
*** dmorita has joined #openstack-swift03:14
*** sams-gleb has joined #openstack-swift03:18
*** dmorita has quit IRC03:18
*** dmorita has joined #openstack-swift03:20
*** dmorita has quit IRC03:22
*** sams-gleb has quit IRC03:22
*** dmorita has joined #openstack-swift03:22
*** zaitcev has quit IRC03:22
*** zaitcev has joined #openstack-swift03:23
*** ChanServ sets mode: +v zaitcev03:23
*** dmorita_ has joined #openstack-swift03:24
*** dmorita has quit IRC03:26
*** dmorita_ has quit IRC03:28
*** links has joined #openstack-swift03:37
*** dmorita has joined #openstack-swift03:39
*** dmorita has quit IRC03:44
*** m_kazuhiro has quit IRC03:45
*** zhurong has quit IRC03:48
*** dmorita has joined #openstack-swift04:05
*** dmorita_ has joined #openstack-swift04:07
*** dmorita has quit IRC04:07
*** dmorita has joined #openstack-swift04:08
*** dmorita__ has joined #openstack-swift04:10
*** dmorita_ has quit IRC04:11
*** dmorita has quit IRC04:13
*** dmorita__ has quit IRC04:15
*** chsc has joined #openstack-swift04:19
*** chsc has joined #openstack-swift04:19
*** chsc has quit IRC04:26
*** psachin has joined #openstack-swift04:28
kota_good afternoon04:31
mattoliveraukota_: o/04:34
mahatickota_: mattoliverau o/ morning04:36
mattoliveraumahatic: good morning04:36
kota_mahatic: mattoliverau: o/04:38
kota_mattoliverau: thanks for your input for composite ring CLI etherpad, I was looking at it briefly in the last night04:38
mattoliverauno problem :) I feel both sides have pros and cons.. and I'd like to see it all apart of ringbuilder, but feel its better to move forward then get stuck at this juncture, but that might just be my own opinion :)04:40
mattoliveraubut worried whatever we add to ringbuilder now, will have to stay for a while, so zipper makes sense in the short term04:41
mattoliverauuntil we can get it right04:41
mattoliverauanyway thats what my comments boiled down to04:41
kota_mattoliverau: sounds good opinion, thanks.04:42
*** dmorita has joined #openstack-swift04:54
*** Jeffrey4l_ has quit IRC04:55
*** Jeffrey4l has joined #openstack-swift04:56
*** gkadam has joined #openstack-swift04:57
*** dmorita_ has joined #openstack-swift04:58
*** dmorita has quit IRC04:59
*** dmorita_ has quit IRC05:00
*** dmorita has joined #openstack-swift05:00
*** dmorita_ has joined #openstack-swift05:01
*** dmorita has quit IRC05:01
*** dmorita_ has quit IRC05:02
*** dmorita has joined #openstack-swift05:03
*** dmorita_ has joined #openstack-swift05:04
*** dmorita__ has joined #openstack-swift05:05
*** dmorita_ has quit IRC05:05
*** dmorita has quit IRC05:07
*** dmorita__ has quit IRC05:10
*** dmorita has joined #openstack-swift05:12
*** dmorita_ has joined #openstack-swift05:16
*** dmorita has quit IRC05:17
openstackgerritJeremy Zhang proposed openstack/swift master: Using assertIsNone() instead of assertEqual(None)  https://review.openstack.org/44889405:17
*** JimCheung has joined #openstack-swift05:17
*** sams-gleb has joined #openstack-swift05:20
*** JimCheung has quit IRC05:21
*** adriant has quit IRC05:23
*** dmorita_ has quit IRC05:24
*** sams-gleb has quit IRC05:25
*** dmorita has joined #openstack-swift05:36
*** dmorita_ has joined #openstack-swift05:38
*** dmorita has quit IRC05:38
*** dmorita_ has quit IRC05:42
*** dmorita has joined #openstack-swift05:43
*** dmorita_ has joined #openstack-swift05:44
*** dmorita has quit IRC05:44
*** dmorita_ has quit IRC05:46
*** dmorita has joined #openstack-swift05:46
*** dmorita has quit IRC05:47
*** dmorita has joined #openstack-swift05:47
*** dmorita_ has joined #openstack-swift05:49
*** dmorita has quit IRC05:49
*** dmorita_ has quit IRC05:50
*** dmorita has joined #openstack-swift05:50
*** dmorita has quit IRC05:51
*** dmorita_ has joined #openstack-swift05:52
*** zhurong has joined #openstack-swift05:53
*** dmorita_ has quit IRC05:56
openstackgerritJeremy Zhang proposed openstack/swift master: Using assertIsNone() instead of assertEqual(None)  https://review.openstack.org/44889406:04
*** tonyb_ has joined #openstack-swift06:06
*** mlanner_ has joined #openstack-swift06:12
*** tonyb has quit IRC06:13
*** swifterdarrell has quit IRC06:13
*** mlanner has quit IRC06:13
*** mlanner_ is now known as mlanner06:13
*** swifterdarrell has joined #openstack-swift06:13
*** ChanServ sets mode: +v swifterdarrell06:13
openstackgerritJeremy Zhang proposed openstack/swift master: Using assertIsNone() instead of assertEqual(None)  https://review.openstack.org/44889406:30
*** jaosorior has joined #openstack-swift06:30
*** ChubYann has quit IRC06:32
openstackgerritKazuhiro MIYAHARA proposed openstack/swift master: Another proposal of Automated Tiering's hidden account name rule  https://review.openstack.org/44894106:46
openstackgerritKazuhiro MIYAHARA proposed openstack/swift master: Another proposal of Automated Tiering's hidden account name rule  https://review.openstack.org/44894106:51
*** dmellado has quit IRC07:03
*** pdardeau_ has quit IRC07:03
*** mmmucky has quit IRC07:04
*** pdardeau_ has joined #openstack-swift07:04
*** mmmucky has joined #openstack-swift07:05
*** dmellado has joined #openstack-swift07:08
*** bkopilov has joined #openstack-swift07:18
*** sams-gleb has joined #openstack-swift07:22
*** sams-gleb has quit IRC07:26
*** dmorita has joined #openstack-swift07:38
*** dmorita_ has joined #openstack-swift07:39
*** dmorita has quit IRC07:43
*** amoralej|off is now known as amoralej07:43
*** dmorita_ has quit IRC07:43
*** tesseract has joined #openstack-swift07:44
*** sams-gleb has joined #openstack-swift07:50
*** dmorita has joined #openstack-swift07:52
*** dmorita has quit IRC07:56
openstackgerritjunboli proposed openstack/python-swiftclient master: remove py34 from envlist and its reference from classifier in setup.cfg  https://review.openstack.org/44897807:56
*** pcaruana has joined #openstack-swift08:06
*** PavelK has joined #openstack-swift08:13
*** Administrator_ has quit IRC08:20
*** Administrator_ has joined #openstack-swift08:21
*** dmorita has joined #openstack-swift08:21
*** dmorita_ has joined #openstack-swift08:22
*** dmorita__ has joined #openstack-swift08:24
*** dmorita_ has quit IRC08:24
*** dmorita has quit IRC08:24
*** dmorita__ has quit IRC08:28
*** dmorita has joined #openstack-swift08:33
acolesgood morning08:34
*** oshritf_ has joined #openstack-swift08:37
*** dmorita has quit IRC08:38
acolesrledisez: did you find what you need re wsgicontext?08:38
rledisezacoles: yes, i think so. it looks like i need to fix the tests after using wsgicontext. i'll look into it today08:40
acolesrledisez: ah, that's the problem with tests :) let me know if you have any more questions08:43
*** geaaru has joined #openstack-swift08:44
acolesrledisez: BTW, the issue w.r.t. duplicate header names/nit putting headers into a dict - I should have said that swift does not produce any duplicate header names, but in the past I have heard of third party middlewares that do. In fact, that's how i learnt about this - one of my first swift patches did exactly this with response headers and broke someone's middleware :/08:45
rledisezacoles: sure. no matter what, we should respect RFC when possible :)08:45
*** cbartz has joined #openstack-swift08:53
*** jaosorior is now known as jaosorior_brb08:55
*** ma9_ has quit IRC08:56
*** openstackgerrit has quit IRC09:03
*** ma9_ has joined #openstack-swift09:04
*** openstackgerrit has joined #openstack-swift09:07
openstackgerritJeremy Zhang proposed openstack/swift master: Using assertIsNone() instead of assertEqual(None)  https://review.openstack.org/44889409:07
*** ma9_ has quit IRC09:07
acolesmattoliverau: I just added sharding to the Boston topics etherpad so now you must come :)09:14
*** ma9_ has joined #openstack-swift09:15
mattoliverauacoles: well now that its written I have no choice ;)09:20
acolesbook those flights!09:20
*** jordanP has joined #openstack-swift09:21
*** gabor_antal has quit IRC09:23
*** gabor_antal has joined #openstack-swift09:24
*** ma9_ has quit IRC09:30
*** ma9_ has joined #openstack-swift09:33
*** links has quit IRC09:39
*** links has joined #openstack-swift09:56
acolesalecuyer: rledisez: I left some comments on https://etherpad.openstack.org/p/swift-losf-base10:03
openstackgerritMerged openstack/swift master: keep consistent naming convention of swift and urls  https://review.openstack.org/44738710:06
openstackgerritRomain LE DISEZ proposed openstack/swift master: Rewrite redirection in cname_lookup & domain_remap  https://review.openstack.org/43577110:06
*** geaaru has quit IRC10:11
*** links has quit IRC10:11
*** links has joined #openstack-swift10:23
*** geaaru has joined #openstack-swift10:23
openstackgerritAlistair Coles proposed openstack/swift master: Add more doc and test for cors_expose_headers option  https://review.openstack.org/44502110:25
*** links has quit IRC10:35
*** links has joined #openstack-swift10:51
*** jaosorior_brb is now known as jaosorior11:02
openstackgerritKota Tsuyuzaki proposed openstack/swift master: Add Composite Ring Functionality  https://review.openstack.org/44192111:02
*** jistr is now known as jistr|afk11:06
openstackgerritRomain LE DISEZ proposed openstack/swift master: Replace replication_one_per_device by custom count  https://review.openstack.org/39078111:06
*** sams-gleb has quit IRC11:08
*** sams-gleb has joined #openstack-swift11:09
*** sams-gleb has quit IRC11:13
openstackgerritjunboli proposed openstack/swift master: fixed get suffix of filename  https://review.openstack.org/44907711:17
openstackgerritKota Tsuyuzaki proposed openstack/swift master: Eliminate node_index in Putter  https://review.openstack.org/44307211:27
kota_just rebased11:29
* kota_ is saying about patch 44307211:29
*** mvk has quit IRC11:36
*** ntata has quit IRC11:36
*** sgundur- has quit IRC11:36
*** mmotiani has quit IRC11:36
*** sams-gleb has joined #openstack-swift11:52
*** mvk has joined #openstack-swift12:06
*** bkopilov has quit IRC12:12
*** zhurong has quit IRC12:22
*** NM has joined #openstack-swift12:24
*** jistr|afk is now known as jistr12:27
*** catintheroof has joined #openstack-swift12:44
*** klamath has joined #openstack-swift12:46
*** klamath has quit IRC12:47
*** klamath has joined #openstack-swift12:47
*** psachin has quit IRC12:52
kota_acoles: I updated some my patches, will look at your great improvement (e.g. around node election) tommorow, thanks!12:57
kota_and it's time I should leave my office.12:57
kota_bye12:57
acoleskota_: I saw - but you should be sleeping !12:58
kota_;-)12:58
acolesthanks - btw the node iter patches are WIP - I have some more thoughts for improvements on those. but still appreciate any comments on the direction there.12:59
kota_ok12:59
acoleskota_: I am looking at your updated composite builder again today12:59
*** catintheroof has quit IRC13:03
kota_acoles: thx! (now on the way home)13:03
*** gkadam has quit IRC13:04
tdasilvakota_: wow, that's a late night for you!13:05
* acoles was thinking the same13:06
*** amoralej is now known as amoralej|lunch13:07
*** kei_yama has quit IRC13:11
*** silor has joined #openstack-swift13:17
alecuyeracoles: thanks a lot for your comments on the etherpad. I will submit a small patch soon13:27
*** oshritf_ has quit IRC13:35
*** oshritf_ has joined #openstack-swift13:37
*** Italux has joined #openstack-swift14:05
*** Italux has left #openstack-swift14:05
*** vint_bra has joined #openstack-swift14:10
*** ChanServ sets mode: +v tdasilva14:12
*** dmorita has joined #openstack-swift14:18
*** dmorita_ has joined #openstack-swift14:21
*** dmorita_ has quit IRC14:21
*** dmorita_ has joined #openstack-swift14:22
*** dmorita has quit IRC14:22
*** amoralej|lunch is now known as amoralej14:22
acolesalecuyer: hope it helps14:23
*** dmorita has joined #openstack-swift14:26
*** dmorita_ has quit IRC14:28
*** oshritf_ has quit IRC14:29
*** links has quit IRC14:38
*** oshritf_ has joined #openstack-swift14:41
openstackgerritOpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements  https://review.openstack.org/8873614:42
*** _JZ_ has joined #openstack-swift14:58
*** silor has quit IRC15:04
*** vinsh has joined #openstack-swift15:05
*** dmorita has quit IRC15:09
*** vinsh has quit IRC15:19
*** dmorita has joined #openstack-swift15:24
*** dmorita_ has joined #openstack-swift15:28
*** dmorita has quit IRC15:29
*** dmorita_ has quit IRC15:31
*** dmorita has joined #openstack-swift15:32
*** dmorita_ has joined #openstack-swift15:33
*** dmorita has quit IRC15:33
*** vinsh has joined #openstack-swift15:35
*** dmorita has joined #openstack-swift15:37
*** dmorita_ has quit IRC15:38
*** mmotiani has joined #openstack-swift15:38
*** vinsh has quit IRC15:40
notmynamegood morning15:40
*** dmorita_ has joined #openstack-swift15:40
*** dmorita_ has quit IRC15:41
*** dmorita_ has joined #openstack-swift15:41
*** dmorita has quit IRC15:42
*** sgundur- has joined #openstack-swift15:44
*** dmorita has joined #openstack-swift15:45
*** dmorita_ has quit IRC15:46
*** dmorita has quit IRC15:46
*** dmorita has joined #openstack-swift15:46
*** ntata has joined #openstack-swift15:48
*** dmorita_ has joined #openstack-swift15:49
*** dmorita has quit IRC15:50
*** sams-gleb has quit IRC15:50
*** sams-gleb has joined #openstack-swift15:51
*** dmorita_ has quit IRC15:54
*** amoralej is now known as amoralej|brb15:54
*** sams-gleb has quit IRC15:55
*** dmorita has joined #openstack-swift15:57
*** dmorita_ has joined #openstack-swift15:58
*** dmorita_ has quit IRC15:59
*** dmorita_ has joined #openstack-swift15:59
*** dmorita_ has quit IRC16:00
*** dmorita_ has joined #openstack-swift16:00
*** dmorita has quit IRC16:01
*** dmorita_ has quit IRC16:05
*** silor has joined #openstack-swift16:05
*** chsc has joined #openstack-swift16:07
*** chsc has joined #openstack-swift16:07
*** bkopilov has joined #openstack-swift16:11
*** dmorita has joined #openstack-swift16:24
*** JimCheung has joined #openstack-swift16:25
*** sams-gleb has joined #openstack-swift16:25
*** dmorita has quit IRC16:27
*** dmorita has joined #openstack-swift16:27
*** SkyRocknRoll has joined #openstack-swift16:33
notmynameIRC is less lonely when you don't filter the join/part messages16:36
rledisezIRC is never lonely, just need to trigger reactions16:39
rledisezgood morning :)16:39
*** amoralej|brb is now known as amoralej16:41
*** oshritf_ has quit IRC16:47
notmynamebut now that I killed patchbot, I can't inadvertently trigger nonsense reactions!16:47
timburkegood morning16:53
acolesI miss patchbot16:54
rledisezi miss reading clayg complaining about patchbot ;)16:55
timburke:]16:55
notmynametimburke: ...and nothing happened16:56
timburkenotmyname: what? i was just smiling :P16:56
acoleslast ain't what it used to be16:57
*** cbartz has left #openstack-swift16:58
*** dmorita has quit IRC17:01
*** dmorita has joined #openstack-swift17:02
*** catintheroof has joined #openstack-swift17:07
notmynameplease continue to add stuff to https://etherpad.openstack.org/p/BOS-Swift-brainstorming17:08
openstackgerritPavel Kvasnička proposed openstack/swift master: Container/account disk drive error results double space usage on rest drives  https://review.openstack.org/44848017:14
*** pcaruana has quit IRC17:14
*** mvk has quit IRC17:17
*** gyee has joined #openstack-swift17:20
*** PavelK has quit IRC17:27
*** jaosorior has quit IRC17:30
*** tonanhngo has joined #openstack-swift17:30
*** klrmn has joined #openstack-swift17:31
*** gyee has quit IRC17:33
*** gyee has joined #openstack-swift17:34
*** gyee has quit IRC17:39
tdasilvaI've been looking at this file and can't quite figure out what's going on, maybe somebody can help me: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/swift.yaml17:47
tdasilvathat first job (line 2) is to run the swift functional tests17:47
tdasilvabut i think we run them against swiftclient patches???17:48
tdasilvaam i reading this wrong?17:48
acolestdasilva: there are two templates - one for swift then one for swiftclient17:48
tdasilvaright17:49
acolesso one is used in swift gate job and the other in swiftclient gate jobs17:49
acolesAFAIK they are reference by the name attr17:50
timburketdasilva: yeah, i remember noticing us running swift dsvm for swiftclient too :-/ but iirc swift *does* import swiftclient for func tests, so maybe there was a reason for it?17:50
tdasilvaacoles, timburke: right, so that's what I noticed17:51
timburkehttps://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L16625-L1663617:51
tdasilvathe swift func tests are run both on swift and swiftclient patches17:51
tdasilvaand the swiftclient tests are run only against the swiftclient patches17:51
*** links has joined #openstack-swift17:52
tdasilvawhat does dsvm stand for?17:52
*** mvk has joined #openstack-swift17:53
timburkedevstack vm as i recall17:53
acolesdo we do that so that swiftclient change can't cause swift func tests to break?17:54
timburkei think the swift tests are intended to ensure so we don't break get_auth for  https://github.com/openstack/swift/blob/2.13.0/test/functional/swift_test_client.py#L29 (but i could certainly be wrong)17:54
acolesi.e. run the swift dsvm job against swiftclient patches17:54
timburkei'm not sure the dsvm job actually *covers* that line, and even if it did, it seems like overkill; something more like gate-swift-tox-xfs-tmp-func-ubuntu-xenial should be perfectly sufficient and run in 1/6 the time17:56
tdasilvaoh, i see, so even thou it is running the swift func tests, that swiftclient patch is what gets installed on the node for the test17:56
timburkemaybe *that's* why we went with dsvm -- to ensure the in-review version gets installed...17:56
tdasilvamaybe17:57
tdasilvahehe17:57
tdasilvatimburke: which actually gives me another piece to the puzzle17:57
tdasilvathe reason i'm asking all this is because i'd like to have a job that tests libec and pyeclib together17:58
acolestdasilva: I think that's the goal - check swift func tests pass with the patched version of swiftclient17:58
*** links has quit IRC17:58
tdasilvaacoles: right17:59
timburkei'd be all for paring down the services that get brought up as part of that, though. we've already got gate-tempest-dsvm-neutron-src-python-swiftclient-ubuntu-xenial for cross-project coverage, right? seems unnecessary to bring up all of devstack when all we really want is swift (and maybe keystone) to ensure we don't break swift func tests...18:00
timburketdasilva: yeah... i think there's some devstack-y goodness to make everything work right with Depends-On and all of that... not sure where the magic is, though18:01
tdasilvatimburke: ack, will keep playing with it18:04
tdasilvaunfortunately i can't find much documentation so have to rely on trial-and-error18:04
timburketdasilva: yeah, i've always had that feeling when looking at devstack... it's a little hard to trace where things happen and what affects a change will have...18:06
*** catintheroof has quit IRC18:07
*** ujjain has quit IRC18:13
acolestdasilva: timburke devstack also helpfully installs (a subset of) the swift test projects/users into keystone so func tests can run18:13
tdasilvaacoles: oh nice, so this is probably the reason for this particular job:18:15
tdasilvahttps://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/swift.yaml#L4518:15
acolesyeah and so that swiftclient func tests use a similar set of test accounts as swift tests, see https://github.com/openstack/python-swiftclient/blob/2a62dd88ff48c2e281679c470d47dc72c82707b6/tests/sample.conf18:18
tdasilvaacoles: but what about this line here:18:20
tdasilvahttps://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/swift.yaml#L7818:20
tdasilvait's actually referring to the sample.conf in the swift dir18:20
tdasilvanot the swiftclient dir18:20
tdasilvaplus, is that sample.conf modified when it gets installed in devstack? because the default is v118:21
tdasilvanot v218:21
*** jordanP has quit IRC18:21
acolesyes, so here https://github.com/openstack/python-swiftclient/blob/2a62dd88ff48c2e281679c470d47dc72c82707b6/tests/sample.conf you see the func tests are actually run twice, once using a test file written by devstack IIRC (/etc/swift/test.conf) and once using the sample file in tree.18:26
acolestdasilva: for example here you'll see one run has a bunch of skips, most likely for v2 only tests when run against v1. http://logs.openstack.org/04/443104/4/check/gate-swiftclient-dsvm-functional-ubuntu-xenial/780af79/console.html18:27
acoleshmmm, no line by line test results in the logs18:27
acolessame happen with the swift func tests18:28
tdasilvaacoles: got it18:32
openstackgerritTim Burke proposed openstack/swift master: Drop support for auth-server from common/manager  https://review.openstack.org/44927018:37
*** patchbot has joined #openstack-swift18:37
*** patchbot has quit IRC18:38
notmynametdasilva: are you at vault this week?18:47
*** geaaru has quit IRC18:49
*** tesseract has quit IRC19:28
notmynamehas anyone gotten slqite to work on osx?19:36
notmyname*sqlite19:36
*** amoralej is now known as amoralej|off19:46
*** ChubYann has joined #openstack-swift19:48
claygnotmyname: since when does sqlite not work on osx!?19:51
notmynameclayg: no idea!19:51
notmynamehttps://gist.github.com/notmyname/dff0c20eee60ef1a1190e0c02be3f36819:52
*** SkyRocknRoll has quit IRC19:52
claygnotmyname: so i installed python from brew I think?  https://gist.github.com/clayg/7e5c64c3d99d88b9fadeadcf5cdeca4319:55
notmynameclayg: what versuon of osx do you have?19:55
notmynamehmm, I've got a 16.4.0 kernel19:56
notmynameit's not too important. just inconvenient19:56
*** NM has quit IRC19:56
claygoh yeah *thats* probably it - Version 10.12.2 - Sierra?19:57
claygthe updates store w/e says I'm supposed o restart and get 10.12.3 - no idea how long I've been putting that off19:57
notmynamehmm19:58
notmynameI'm on 10.12.319:58
notmynameoh well. like I said, not critical, just inconvenient19:58
claygwell I have no idea how to make mac's built in python do *anything* useful19:58
claygso I install python from brew and go from there - AFAIK that's just "how it's done"19:58
notmynamesurely you mean "ssh to a vm or remote box running linux" right?19:59
claygyou're the one trying to make python do something useful on your mac19:59
notmynamelol20:00
notmynameand my solution now is to add an scp step to copy my python file to a remote vm running linux!20:00
claygok, well i'm repeating myself a third time - but your other option is to install python with brew and use that primarlly for dev/python stuff20:01
claygbut your other option works well too!  so don't listen to me!20:01
notmynameactually this is all your fault!20:01
notmynameI killed patchbot (your fault), and now i'm looking at sopel (again, your fault), and sopel uses sqlite3 (that part isn't your fault)20:02
notmyname;-)20:02
notmynameWINKY FACE!20:02
timburkenotmyname: `pip install sqlite3`?? i thought that was stdlib! https://docs.python.org/2/library/sqlite3.html20:02
notmynameOUTRAGE! INJUSTICE! (and my kids' favorite new word) PREPOSTEROUS!20:03
timburkenotmyname: literally *every* version of python i have is happy to import it: http://paste.openstack.org/show/603979/20:05
*** NM has joined #openstack-swift20:06
tdasilvanotmyname: I didn't go to vault this year20:12
notmynametdasilva: is this like you not going to the boston summit? isn't it *right there*?20:13
tdasilvanotmyname: heh, at least the summit I get a free pass20:13
notmyname:-)20:13
tdasilvavault was like $700 or something20:13
notmynameouch20:13
tdasilvaplus hotel and all that20:14
tdasilvait's a bit of a hike for me to drive in...20:14
notmynameyou don't get to expense a week of mortgage for a conference ;-)20:14
tdasilvaprobably more than 2hrs in the morning20:14
notmyname(although that would be awesome for local events)20:14
timburkebetween https://review.openstack.org/#/c/232162/53/swift/common/middleware/symlink.py@141 and https://review.openstack.org/#/c/423906/7/swift/common/request_helpers.py@717 and https://github.com/openstack/swift/blob/2.13.0/swift/common/middleware/crypto/decrypter.py#L350 ... what do people think about moving a/c listing response-type mangling to proxy middleware, and having the proxy *always* do `?format=json` during requests? and pro20:15
timburkebably removing xml support from backend servers in the process20:15
notmynametimburke: in general, I'm a fan of the storage nodes only doing one kind of response and the proxy mutating it for the client20:16
timburkenotmyname: yeah, i'd kinda like to get rid of txt responses, too, but that'd probably break some internal clients. xml seems safe to get rid of, though?20:17
notmynametimburke: from the storage node, yeah. and we should look into the internal client usage.20:18
jrichlitimburke: so that means that there must be a required mw that will make the necessary transformation.  Both symlinks and encryption are optional.  where would the required transformation live?20:19
jrichlior would it take place in the proxy controller or somewhere else in the proxy?20:20
timburkejrichli: i was thinking far left; just right of cache or that first proxy-logging20:20
jrichliok, so you are thinking a new required mw?20:21
notmynametimburke: in the proxy controller itself might not be a bad idea20:21
notmynameinstead of middleware20:21
timburkewith the container controller still forcing a json listing20:21
timburkenotmyname: proxy controller doesn't help the encryption & symlink cases, though. i want middleware to only need to think about one format20:21
*** ujjain has joined #openstack-swift20:22
*** ujjain has joined #openstack-swift20:22
jrichlioic.  so one format to worry about in the response.  but if you have both symlinks and encryption, wouldn't symlinks see the already processed response (xml)?  so would still need to worry about all options?20:24
tdasilvatimburke: and would that middleware know about symlink and encryption changes?20:25
jrichlii am a little confused how we things will work if we don't only have one spot to transform from the one obj-server format.20:26
timburketdasilva: not if we do it right :-)20:26
jrichlioh - so like a method they all add to?20:26
tdasilvatimburke: so each middleware would still need to iterate over the list?20:26
timburkejrichli: with both symlinks and encryption, there'd be some extra json deserialization/serialization (so each could do their processing), but punt to the new mw for xml formatting20:26
timburketdasilva: ya20:27
jrichlioh, right.  i see.20:27
jrichlithe new mw is the one spot20:27
openstackgerritAlistair Coles proposed openstack/swift master: WIP use composite ring metadata to prevent bad things  https://review.openstack.org/44931020:28
tdasilvatimburke: wouldn't we gain by having just one place doing any changes necessary to the listing? I guess that would not be good for third-party middleware20:28
jrichlimakes sense.  a middleware to handle listing formats.20:28
tdasilvabut just trying to save on multiple mws traversing through the same list multiple times20:28
jrichlitrue: third-party middleware could be a concern.  other than that, i think it makes a lot of sense to encapsulate listing format into a mw.20:29
timburketdasilva: either we have each middleware handle its own list iteration, or we create some new, non-WSGI callback pipeline. the former seems preferable to having a new copy_hook look-alike?20:32
acoleskota_: I wrote some *early* ideas for composite ring metadata in patch 44931020:34
acolesgood night20:34
timburkeg'night acoles!20:34
tdasilvaright, that's sort of i was thinking. e.g., encryption provides the callback function to decrypt data, but that gets called from this new mw20:34
tdasilvaacoles: good night!20:34
acolescheers guys, have a good evening20:34
jrichligood night acoles20:35
timburkejrichli: agreed, third-party middleware may be a problem. but we've broken them before for less-obvious wins. and i *think* most middlewares that do their own container listings would typically go for json anyway? certainly versioned_writes and dlo do20:36
jrichlitdasilva: I guess that would be assuming that processing is a line-by-line independent thing.  probably true now, but it technically adds that restriction.  unless it was still ok for a mw to do its own loop20:38
*** silor1 has joined #openstack-swift20:44
*** silor has quit IRC20:46
*** silor1 has quit IRC20:49
*** clarkb has joined #openstack-swift21:00
*** JimCheung has quit IRC21:01
clarkbHello. I have been looking at memory consumption of various portions of our integration testing ebcause OOMs have become a problem in some gate jobs. One thing I noticed was that our python35 integration job (which doesn't run swift) uses significantly less memory than the job which does use swift21:01
clarkbdigging into that more it appears that we have workers + 1 processes for most of the swift processes (proxy, container server, object server, etc)21:01
clarkbfor the container server and object server this is 3 processes each as they have 2 workers. My question is can that be dialed back to 1 worker safely without losing test coverage or degrading the service?21:02
clarkband potentially is there a way to drop the extra fork and just have N processes instead of N+1?21:02
clarkbnotmyname: clayg ^21:02
notmynameclarkb: most likely, yes21:02
notmynameclarkb: the "workers" setting in the config files21:03
clarkbnotmyname: ya I see that, so it should be fine to set that down to 1 for container server and object server?21:04
notmynameyeah21:04
clarkboh and account server. Cool I can get that pushed up then. Thanks21:04
clarkbas for the N+1 situation I'm guessing thats beacuse the parent is the coordinator of a bunch of workers? so you can't just get rid of it?21:04
*** JimCheung has joined #openstack-swift21:05
notmynamewe might could. but TBH it's not what's used in any prod environment, so I haven't spend much (any?) time thinking about that21:05
clarkbnotmyname: prod would use the N+1 regardless? or prod uses a completely different process management setup?21:06
notmynameprod would use N+1. I don't know of any prod setups with only one core per server. we want at least one worker per core21:07
clarkbgotcha21:07
claygzaitcev: another thing to think about if get get to make a hard cutover to a v2 internal storage api21:14
claygwe should design them so that a/c/o apis can all run on the same port/worker pool21:14
claygbe useful for the swift-in-the tiny use-case - and not terrible/unreasonable for a sort of thought out api21:15
*** NM has quit IRC21:15
zaitcevuse /av2 /cv2 /ov1 or a header?21:15
clayg^ cc redbo /<v>/dev/<service>/part/21:15
claygNOT header ;)21:15
zaitcevheh21:15
claygi'm sick of headers that dramatically change the nature of the api - stupid x-backend-storage-policy-index21:16
clayg/<v>/dev/<service>[-spi]/part/ maybe21:16
*** NM has joined #openstack-swift21:18
jrichliQuestion on SAIO at https://docs.openstack.org/developer/swift/development_saio.html#common-post-device-setup21:23
jrichliThe /etc/rc.local file is modified, but nothing is explicitly run to invoke that file.  I know a reboot runs it, but what in the instructions are supposed to cause that file to be run?  is the next call to 'service' supposed to run it?21:23
jrichliThis is related to patch https://review.openstack.org/44316221:24
zaitcevThere have to be limits of what we can teach people about basics of Linux.21:25
zaitcevIt's like these days they teach students to write in complete sentences in colleges, because they enjoyed self-esteem improvements in K-12 systems instead of studying.21:26
notmynamemaybe it's annoying to add a "source rc.local" comment because "kids these days...", but there's always new people learning stuff (and sometimes even older people learning stuff)21:28
clarkbspeaking of learning things, on the N+1 with N == numcpu, do you not tend to oversaturate IO that way?21:29
clarkbseems like you'd run out of pci lanes before you did cpu cycles in that setup21:29
notmynameclarkb: normally there's a lot more drives than cores, and disk IO being what it is, a thread doing stuff on the disk is blocked. so having more workers allows us to handle more requests while the other requests are busy spinning platters21:32
notmynameclarkb: and because python and eventlet, we need to fork child processes (hello, golang!)21:32
notmynameclarkb: you are hitting a very important port. or raising a very important question. "how does swift schedule IO to make most efficient use of all of the available disk iops"21:34
claygjrichli: it's a sysV thing - I'm sorta glad you brought it up because I've never really considered how systemd and rc.local get along21:34
clarkbnotmyname: ya I recall the email explaining how golang was useful around this21:35
notmynameclarkb: and the answer is "meh. but we want to get better and have some reasonable ideas on how to get there." clayg's working on some great stuff about scaling the background daemons to do better in relation to this21:35
clarkbclayg: occasionally they don't get along and then things break in weird places (like puppet)21:36
clarkbclayg: but usually you jsut drop fiels in the usual places and reload the systemd config and can start and stop services like any other systemd unit21:36
notmynamejrichli: I think you mean ^21:36
claygclarkb: yeah I think the answer to my question was "systemd tries to be backwards compatible with crufty bits like that - you should test it!"21:36
notmynamebah! sorry.21:37
notmynameboth clarkb and clayg are the same color in my irc client (and very similar)21:37
clarkb:)21:38
notmynameclarkb: there's a lot of facets to the solution for better iops. much of it is language agnostic (determining the work to be done and scheduling it effectively) and some isn't (what's in the data path for moving bits from nic to disk and back)21:38
notmynamehttp://d.not.mn/very_confusing_colors.png21:39
*** patchbot has joined #openstack-swift21:46
notmynamepatch 123421:46
patchbothttps://review.openstack.org/#/c/1234/ - tempest - Removing link doctoring in test_images (MERGED)21:46
notmynameok, a *limited* version of patchbot is back21:46
jrichliyay!  welcome back patchbot21:47
*** NM has quit IRC21:47
claygnotmyname: thanks bro - sorry to be an ass - but you're awesome21:47
jrichlilimited how?  can I do all the poking now to see what happens?21:47
claygpatchbot: do you hate me?21:47
notmynamelimitations include (1) not being drunk (2) not responding to patch URLS (3) only responding to one patch number per line21:47
claygnotmyname: timburke will fix that eventually - you rock21:48
jrichlii can live with that21:48
notmynamelol21:48
timburkelast time patchbot was here, he seemed to hate clayg...21:48
timburkeyay! :]21:48
claygtimburke: mutual21:48
claygthis new cat sems cool so far21:48
notmynameI want the multi-response-per-line thing21:48
claygyeah that was not pretty not terrible :\21:48
notmynamepatch 1234 patch 567821:49
patchbothttps://review.openstack.org/#/c/1234/ - tempest - Removing link doctoring in test_images (MERGED)21:49
timburkenotmyname: still gonna live at https://github.com/notmyname/Patches/ ?21:49
notmynamep 654321:49
patchbothttps://review.openstack.org/#/c/6543/ - keystone - Just for testing (ABANDONED)21:49
notmynametimburke: dunno21:49
notmynameI gotta say that I *love* sopel relative to supybot though21:50
notmynametimburke: I'll get it up somewhere and let you know21:50
timburkenotmyname: that's the part i really care about :-)21:50
timburkeand do links like https://review.openstack.org/#/c/446185/ still work?21:51
timburke:-(21:52
mattoliverauMorning21:52
notmynametimburke: that's the other thing I want to get working. that one should be close to trivial21:52
jrichlitimburke: that was limitation (2) ;-)21:52
timburkejrichli: fah! reading!21:53
jrichlilol21:53
notmynameI'm thinking of adding a "if time.now() == 3am: bot.tell(clayg, "I'm still watching you...")21:53
jrichlimismatched quotes21:54
jrichli:-)21:54
clayglol @ jrichli21:54
claygnotmyname: no funny21:54
jrichliI know timburke is dying to test some more21:55
timburkenotmyname: you've gotta get some randomness in there. like, spin up a separate thread to sleep(random.randint(2, 8) * 3600 + random.randint(0, 3599)) then pick one of at least a dozen messages to tell21:58
*** dmorita has quit IRC21:58
notmynameof course. it was just pseudocode ;-)21:58
timburkebonus points for doing it in some random channel clayg is known to frequent rather than a pm21:58
clayg:D21:59
claygI have the best friends21:59
* clayg 's heart grows three sizes22:00
*** patchbot has quit IRC22:02
timburke:-(22:02
timburkepatchbot, i miss you already!22:02
notmyname...testing...22:03
zaitcevNo, better use an unbalanced '22:09
zaitcevMy height is 6'5"22:10
*** sams-gleb has quit IRC22:42
*** patchbot has joined #openstack-swift22:43
notmynamezaitcev: how tall are you?22:43
notmynamep 1234 p 567822:43
patchbothttps://review.openstack.org/#/c/1234/ - tempest - Removing link doctoring in test_images (MERGED)22:43
patchbothttps://review.openstack.org/#/c/5678/ - nova - Check that DescribeInstance works with deleted image (MERGED)22:43
notmynamehttps://review.openstack.org/#/c/5678/ https://review.openstack.org/#/c/56799/22:43
patchbotpatch 5678 - nova - Check that DescribeInstance works with deleted image (MERGED)22:43
patchbotpatch 56799 - nova - Compact pre-Icehouse database migrations <= 170. (MERGED)22:43
zaitcev6'5"22:43
notmynameno errors from patchbot!22:43
zaitcevhmm. Needs a command, I suspect.22:43
timburkewooo!22:43
*** sams-gleb has joined #openstack-swift22:43
notmynamepatch 1234 https://review.openstack.org/#/c/5678/22:44
patchbotpatch 1234 - tempest - Removing link doctoring in test_images (MERGED)22:44
notmynamemixing the two forms doesn't work22:44
zaitcevWearing a nicotine patch at 6'5"22:44
*** sams-gleb has quit IRC22:44
zaitcev(which I'm not, but)22:44
timburkep 1234 patch 456722:44
patchbothttps://review.openstack.org/#/c/1234/ - tempest - Removing link doctoring in test_images (MERGED)22:44
patchbothttps://review.openstack.org/#/c/4567/ - python-keystoneclient - Fix ec2-credentials-list to print 'access'. (ABANDONED)22:44
timburkeppp 123422:44
timburkegood22:45
notmynametimburke: https://github.com/notmyname/Patches222:47
notmynametimburke: and you've got commit access there22:48
timburkep qwer22:51
*** tonanhngo has quit IRC22:51
timburkep -122:52
timburkep 022:53
patchbotAttributeError: 'NoneType' object has no attribute 'get' (file "/home/ubuntu/.sopel/modules/patchbot.py", line 39, in get_response)22:53
mattoliverautimburke: you did it22:53
timburkeyay!22:53
timburkenotmyname: ...is that a... p-0 bug?22:54
timburke:P22:54
*** ujjain has quit IRC22:57
*** tonanhngo has joined #openstack-swift22:58
timburketdasilva, jrichli (and i suppose mattoliverau may be interested): fwiw, functests pass with something like https://gist.github.com/tipabu/513dfeabe471e652d17474424328bc74 in the proxy pipeline...22:59
*** ujjain has joined #openstack-swift23:01
*** ujjain has joined #openstack-swift23:01
*** tonanhngo has quit IRC23:02
*** catintheroof has joined #openstack-swift23:13
notmynametimburke: bah23:16
*** catintheroof has quit IRC23:20
*** patchbot has quit IRC23:20
*** patchbot has joined #openstack-swift23:20
notmynametimburke: patch 023:21
*** catintheroof has joined #openstack-swift23:21
*** klamath has quit IRC23:23
claygtimburke: that was *after* you got to look at the source code - all bugs are shallow23:25
claygtimburke: wtf!?   https://gist.github.com/tipabu/513dfeabe471e652d17474424328bc74 is *awesome* - have you had that laying around?!23:26
claygI mean, don't get me wrong - i didn't understand wtf it was doing for like the first 5 mins - but ... well whatever docstrings - psffttt23:27
timburkeclayg: ...i just wrote it. like, today. no tests (beyond swift's func tests) and no immediate plans to write them (because otherwise it's just time taken away from getting sharding loaded into my head)23:28
timburkebut it seemed like we were hitting issues with this sort of thing a decent bit (crypto, symlinks, sharding...) so i figured i'd take a stab at it. if mattoliverau thinks it'll make his life easier with sharding, i could write some tests and we could get a chain going...23:29
clayg*gotcha* - nice work23:30
*** kei_yama has joined #openstack-swift23:31
mattoliverautimburke: so yeah, taking the listing away from the account and container server (and proxy in the case of sharding) is kinda cool. So what all we'll really care about is getting stuff in json then :)23:33
*** catintheroof has quit IRC23:33
mattoliverauonly downside I see, which isn't much of one. If that json listings means more data then the default text listings. However, I assume json listing are very common anyway, and sharding will always use em.23:34
mattoliverauthat is assuming if we have a listing format middleware, we could remove the code from the container/account server. If you want to get non json from them, then just add the middleware to thier pipline too.23:37
*** catintheroof has joined #openstack-swift23:38
mattoliverau\o/ after 272 runs I finally have a second stable/newton swift failure (first one is already up as a backport).23:41
notmynamemattoliverau: nice!23:41
*** sams-gleb has joined #openstack-swift23:44
*** catintheroof has quit IRC23:45
*** sams-gleb has quit IRC23:49

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!