Friday, 2015-07-31

*** haomaiwa_ has quit IRC00:01
*** haomaiwang has joined #openstack-swift00:02
claygwe're the highest of A's00:06
*** nadeem has quit IRC00:08
claygAfter g = itemgetter(2, 5, 3), the call g(r) returns (r[2], r[5], r[3])00:12
clayg^ I DID **NOT** KNOW THAT!?00:13
claygpeluse: ^ who found that!?00:14
*** ccavanna has joined #openstack-swift00:19
*** ccavanna_ has joined #openstack-swift00:22
*** dmorita has joined #openstack-swift00:25
*** ccavanna has quit IRC00:26
peluseyou mean my use of itemgetter in that patch?00:29
peluseif so that'd be me (with a little help from my friend google)00:31
*** jasondotstar has quit IRC00:33
claygpeluse: I think I could use some unittests in unit.obj.test_server that describe the use and behiavor of this object server api wrt to the new x-backend-preferred-frag-index request header and the x-backend-obj-alt-frag-index response header00:35
claygpeluse: I know that ultimately it's the diskfile that's doing all the work and the object server is just brokering them in - but I think it'd be a little easier to dive in if I could make a little black box model of the object server in my head before I dive into the proxy handling00:36
peluseclayg, OK, will work on that tomorrow00:36
claygpeluse: e.g. if an object server has 2, but the proxy says prefer 1 - it will respond 404 - but does that 404 include the alt-frag-indexes?00:37
pelusegotcha00:37
claygwell i say it will 404, last time I checked a diskfile passed a frag_index would raise if it doesn't have the frag00:37
peluseI can add some tests there pretty easily I think... gotta go flip my chicken on the grill now though :)00:38
claygit's probably also worth considering if we should add the headers for HEAD requests - just for sanity and uniformity and out of band poking00:38
claygsure np - thanks00:39
*** bill_az_ has quit IRC00:40
notmynamestopping in for a bit. looks like I missed some excitement this afternoon00:54
notmynamepeluse: I certainly hope they consider swift to be HA.00:58
notmynamesi1v3r_: what ctennis said about exposing the raw devices to swift is important. that's one of the main reasons to not use RAID volumes in a swift cluster00:59
notmynamesi1v3r_: the other big reason is that RAID rebuild times can take a really long time and seriously affect performance while it's happening01:00
*** haomaiwang has quit IRC01:01
notmynamesi1v3r_: other reasons include the fact that swift doesn't lots of small, random reads and writes, and that can be pretty bad for RAID01:01
notmynamesi1v3r_: and once you have a whole RAID volume fail, then Swift has to work around it anyway, just with a larger failure domain and thus more data movement. so keeping the volumes smaller (ie a single drive), you smooth ou the hit from failed volumes01:02
*** haomaiwa_ has joined #openstack-swift01:02
*** jasondotstar has joined #openstack-swift01:03
notmynamesi1v3r_: also, since drives (and storage servers) tend to dominate the cost of a cluster, it doesn't generally make sense to add another $500 per server to the cluster. that ends up being a noticeable percentage of cost01:03
notmynamesi1v3r_: but beyond all that, sure you could use RAID in swift. there's nothing that would break. I think you'd do better without RAID, though01:05
pelusenotmyname, yeah that's what I was thining too but since I didn't see it mentioned I figured I'd better not "assume" :)01:05
notmynamepeluse: I saw "swift" mentioned in my buffer playback in one of the meeting channels. seems there's an HA group or something. but I couldn't find what they were referring to as the "Swift stuff"01:06
notmynamepeluse: how did that come to your attention?01:16
pelusenotmyname, a mktg guy here is working on a white paper about HA and openstack and asked me for some feedback, it only had cinder stuff mentioned.  but its a paper on 'what needs work' so...01:27
peluseand he referenced that PDF I linked above in the paper and I noticed it didn't mention Swift either01:27
notmynamepeluse: so in general, I don't know anything about what that doc is, who produced it, or who's it for. I only know that there is a group that occasionally has meetings (based on my IRC client highlighting it this afternoon)01:28
peluseheh01:29
pelusenothing to sweat, clearly :)01:29
notmynamewell, it's openstack, so these things tend to turn into "Rules You Must Now Follow or Explain Why You are Not" sort of things.01:30
notmynameso I don't know if it's something to sweat or not ;-)01:30
pelusewe could rename the project to Swift-HA :)01:38
notmynameor "Ha! Swift!"01:39
peluseyes!!01:39
notmynameok, I gota run. I'll be back online later tonight01:39
peluselater, I'm done for the eve...01:40
*** kota_ has joined #openstack-swift01:43
*** ChanServ sets mode: +v kota_01:43
kota_good morning01:44
*** jkugel has joined #openstack-swift01:51
*** haomaiwa_ has quit IRC02:01
*** haomaiwang has joined #openstack-swift02:02
openstackgerritjanonymous proposed openstack/swift: Adding bandit for security static analysis testing in swift  https://review.openstack.org/19639502:08
*** bapalm has joined #openstack-swift02:09
mattoliveraukota_: morning02:14
kota_mattoliverau: morning :)02:14
*** jrichli has joined #openstack-swift02:16
*** bapalm has quit IRC02:19
*** openstackgerrit has quit IRC02:31
*** openstackgerrit has joined #openstack-swift02:31
*** hrou has joined #openstack-swift02:38
kota_tdasilva: around?02:49
kota_oh no, EDT is already 10:50 p.m. I'm too late.02:50
kota_k, let's take a lunch to refresh my head. I might miss something.02:53
*** jasondotstar has quit IRC02:58
*** sanchitmalhotra has joined #openstack-swift02:58
*** jkugel has quit IRC02:58
*** haomaiwang has quit IRC03:01
*** haomaiwang has joined #openstack-swift03:02
*** sakaYK has joined #openstack-swift03:11
*** proteusguy has joined #openstack-swift03:14
*** sanchitmalhotra1 has joined #openstack-swift03:19
*** sanchitmalhotra has quit IRC03:21
*** jrichli has quit IRC03:25
*** h00327910__ has quit IRC03:28
*** sanchitmalhotra has joined #openstack-swift03:36
*** sanchitmalhotra1 has quit IRC03:38
*** proteusguy has quit IRC03:41
notmynamehello, world03:51
*** sanchitmalhotra1 has joined #openstack-swift03:57
*** sanchitmalhotra has quit IRC03:59
pelusenotmyname, is that your first program?03:59
*** haomaiwang has quit IRC04:01
*** haomaiwang has joined #openstack-swift04:02
*** ktsuyuzaki has joined #openstack-swift04:04
*** kota_ has quit IRC04:06
*** kota_ has joined #openstack-swift04:07
*** ChanServ sets mode: +v kota_04:07
*** haigang has joined #openstack-swift04:07
*** ktsuyuzaki has quit IRC04:09
notmynameredbo: there was going to be a "ginger pride" day in san francisco, but it was cancelled http://www.breitbart.com/california/2015/07/30/redheads-gather-in-san-francisco-to-celebrate-ginger-pride-day/04:09
*** kota_ has quit IRC04:11
*** ho has quit IRC04:16
*** ho has joined #openstack-swift04:19
*** ho has quit IRC04:22
*** sanchitmalhotra has joined #openstack-swift04:31
*** ppai has joined #openstack-swift04:31
*** sakaYK has quit IRC04:32
*** sanchitmalhotra1 has quit IRC04:33
*** nakagawamsa has joined #openstack-swift04:34
openstackgerritMerged openstack/swift: Swift documentation for first-time contributors.  https://review.openstack.org/20686104:52
openstackgerritMerged openstack/python-swiftclient: swiftclient: add short options to help message  https://review.openstack.org/20290204:54
openstackgerritMerged openstack/swift: Remove error_suppression_interval, error_suppression_limit options.  https://review.openstack.org/20729604:54
*** sanchitmalhotra1 has joined #openstack-swift04:58
*** haomaiwang has quit IRC05:01
*** sanchitmalhotra has quit IRC05:01
*** haomaiwang has joined #openstack-swift05:02
*** sanchitmalhotra has joined #openstack-swift05:05
*** sanchitmalhotra1 has quit IRC05:07
*** SkyRocknRoll has joined #openstack-swift05:18
*** SkyRocknRoll has joined #openstack-swift05:18
*** kota_ has joined #openstack-swift05:19
*** ChanServ sets mode: +v kota_05:19
*** ig0r_ has joined #openstack-swift05:24
redboI had no idea there was going to be a ginger pride day.  That's what I get for skipping the meetings.05:40
*** hrou has quit IRC06:00
*** haomaiwang has quit IRC06:01
*** 18WAAC7EF has joined #openstack-swift06:02
*** silor has joined #openstack-swift06:03
*** ig0r__ has joined #openstack-swift06:11
*** ig0r_ has quit IRC06:12
*** ppai has quit IRC06:19
*** ig0r_ has joined #openstack-swift06:24
openstackgerritMerged openstack/swift: Fix pep8 E265 warning of hacking 0.10  https://review.openstack.org/20724206:24
openstackgerritMerged openstack/swift: Fix warning pep8 E128 warning of hacking 0.10  https://review.openstack.org/20724306:24
openstackgerritMerged openstack/swift: Fix warning pep8 E128 warning of hacking 0.10  https://review.openstack.org/20724306:24
*** ig0r_ has quit IRC06:24
openstackgerritMerged openstack/swift: Fix pep8 E warning for hacking 0.10  https://review.openstack.org/20723706:24
openstackgerritMerged openstack/swift: Update hacking to 0.10.0  https://review.openstack.org/20597706:24
*** mahatic has joined #openstack-swift06:24
*** ppai has joined #openstack-swift06:28
*** sakaYK has joined #openstack-swift06:29
*** sanchitmalhotra1 has joined #openstack-swift06:43
*** sakaYK has quit IRC06:43
*** sanchitmalhotra has quit IRC06:45
*** sakaYK has joined #openstack-swift06:54
*** SkyRocknRoll has quit IRC06:56
openstackgerritMerged openstack/swift: Support keystone v3 domains in swift-dispersion  https://review.openstack.org/19513106:59
*** 18WAAC7EF has quit IRC07:01
*** haomaiwang has joined #openstack-swift07:01
*** sanchitmalhotra has joined #openstack-swift07:07
*** sanchitmalhotra1 has quit IRC07:09
*** rledisez has joined #openstack-swift07:11
*** silor has quit IRC07:11
*** zaitcev has quit IRC07:24
*** geaaru has joined #openstack-swift07:27
*** SkyRocknRoll has joined #openstack-swift07:27
onovyhi, https://review.openstack.org/#/c/207322/ can someone explain me why py3 gate is failing?07:42
*** sakaYK has quit IRC07:45
haypoonovy: it looks like a test failed on Python 3, http://logs.openstack.org/22/207322/2/check/gate-python-swiftclient-python34/4e8af41/testr_results.html.gz07:50
haypoonovy: the call with prefix_b was not done07:51
*** sakaYK has joined #openstack-swift07:56
*** marzif_ has joined #openstack-swift07:57
*** mahatic has quit IRC07:59
*** haomaiwang has quit IRC08:01
*** haomaiwa_ has joined #openstack-swift08:02
*** sanchitmalhotra1 has joined #openstack-swift08:08
onovybut i'm not changing that file at all08:09
onovyhmm, file is same, but function not08:09
*** marzif_ has quit IRC08:09
*** sanchitmalhotra has quit IRC08:10
acolesgood morning08:13
*** joeljwright has joined #openstack-swift08:16
*** ChanServ sets mode: +v joeljwright08:16
*** jasondotstar has joined #openstack-swift08:19
kota_acoles: good morning08:29
acoleshi kota_08:30
*** jistr has joined #openstack-swift08:32
*** ppai has quit IRC08:37
*** marzif_ has joined #openstack-swift08:40
*** marzif_ has quit IRC08:41
*** sakaYK has quit IRC08:42
*** marzif_ has joined #openstack-swift08:42
*** ppai has joined #openstack-swift08:45
*** ppai has quit IRC08:54
*** sakaYK has joined #openstack-swift08:58
*** jordanP has joined #openstack-swift08:59
*** mahatic has joined #openstack-swift08:59
*** haomaiwa_ has quit IRC09:01
*** aix has joined #openstack-swift09:02
*** haomaiwang has joined #openstack-swift09:02
*** dmorita has quit IRC09:07
*** ppai has joined #openstack-swift09:07
acolespeluse: cool. i don't know how to change 'owner' of patch 207165 back to you. but you and clayg are still the authors afaict09:09
patchbotacoles: https://review.openstack.org/#/c/207165/09:09
*** sakaYK has quit IRC09:12
openstackgerritNakagawa Masaaki proposed openstack/swift: SAIO on Ubuntu 14.04 can not pass unittest.  https://review.openstack.org/20779009:16
*** sanchitmalhotra has joined #openstack-swift09:20
*** sanchitmalhotra1 has quit IRC09:22
*** sakaYK has joined #openstack-swift09:23
acolesnotmyname: looks like there is a 'gap' in the review dashboard, reviews with a +2 but no + vote from jenkins disappear?? e.g. patch 20732209:24
patchbotacoles: https://review.openstack.org/#/c/207322/09:24
*** sanchitmalhotra1 has joined #openstack-swift09:26
*** sanchitmalhotra has quit IRC09:28
tamizh_geekHey cschwede, can you please take a look at the comments in https://review.openstack.org/#/c/204196/ ?09:37
*** joeljwright1 has joined #openstack-swift09:40
*** joeljwright1 has quit IRC09:40
*** joeljwright has quit IRC09:42
onovydon't get it. patch 207322 after recheck, another gate failed :)09:50
patchbotonovy: https://review.openstack.org/#/c/207322/09:50
*** akle has joined #openstack-swift09:51
*** ppai has quit IRC09:51
acolesonovy: looks like same test failed but in py26 rather than py34 job?09:54
acolesonovy: the failing test expects two calls to a method, and cursory inspection suggests those calls are made in separate threads. Clearly nothing to do with your patch!09:56
acolesonovy: maybe an intermittent race bug? i wonder if timburke  can help when he wakes up?09:57
cschwedetamizh_geek: done :)09:57
acolestimburke: patch 207322 failing test_upload_delete_dlo_segments http://logs.openstack.org/22/207322/2/check/gate-python-swiftclient-python26/0dda652/testr_results.html.gz09:58
patchbotacoles: https://review.openstack.org/#/c/207322/09:58
tamizh_geekcschwede: Thanks. Now, do I need to wait for another +2 from a core dev?09:58
*** silor has joined #openstack-swift09:59
mahaticacoles: Hello!09:59
acolesmahatic: hi!10:00
cschwedetamizh_geek: yes, the second core is then going to add another +2 and a +A (approval). of course only if he is fine with the patch too ;)10:00
tamizh_geekcschwede: yeah yeah. thanks :) will be my first patch to openstack!10:01
mattoliverauYay, its cschwede, acoles, and mahatic! Good evening all10:01
cschwedetamizh_geek: great, so welcome and thanks for the contribution to openstack!10:02
cschwedemattoliverau: Hello Matthew!10:02
mahaticacoles: I see that there are unit tests for encrypter/decrypter on PUT POST etc. Could you tell me what the probe test, for instance on PUT object supposed to be doing? ^10:02
mahaticmattoliverau: good evening!10:03
acolesmattoliverau: hey you are still alive?10:03
mahatic:D10:03
mattoliverauYeah, feeling great as of this arvo, been quite a week :)10:04
acolesmattoliverau: are you watching the ashes????10:04
acolesmattoliverau: that will cheer you up :P10:04
mattoliverauacoles: don't remind me, you guys are doing too well this test10:04
mahaticlol10:05
acolesmahatic: so an idea for a probe test for a PUT with encryption would be to PUT an object then get the object file direct from backend and check that it is encrypted, and check all the metadata is encrypted etc.10:06
*** ppai has joined #openstack-swift10:06
acolesmattoliverau: it'll swing back the other way next game, and this one ain't over yet!10:06
acolesmahatic: BUT any such probe test would rely on the encryption middleware being in the proxy pipeline so you'd need to make it check the /info and skip if no encrypter/decrypter10:07
acolesmattoliverau: anyways glad you're feeling better10:08
mahaticacoles:  After setting up encryption, I'm finally seeing the errors and failures that are expected as jrichli pointed!10:08
mahaticacoles: okay. I'll look into that10:08
acolesmahatic: great ! (?) ;)10:08
mahaticacoles: :D well, yes. yay, same failures and errors!10:09
acolesmahatic: so at some point we'll need to think about some kind of "default" or sample config for testing crypto e.g. keys, so that a probe test could independently calculate a cipher text that it expects to see on backend obj servers10:10
acolesmahatic: consistency is a good thing!)10:10
*** moo41 has joined #openstack-swift10:11
*** sakaYK has quit IRC10:12
moo41Hi all. Is anybody here who can give me a hint, how I can make an exception within my middleware component visible? I just get a HTTP 500 back and cannot see anything in the log....10:12
moo41I have already added catch_errors to the begin of the pipeline without any change10:13
*** silor1 has joined #openstack-swift10:13
onovyacoles, thanks10:14
*** SkyRocknRoll has quit IRC10:16
*** silor has quit IRC10:16
*** silor1 is now known as silor10:16
mahaticacoles: I see. So the current unit test, "test_basic_put_req" (in test_encrypter.py) isn't really testing encryption. That is on a TODO10:17
mahaticacoles: When I add encrypt/decrypt check on probe test, it will be similar to unit test with the TODO I think. Or am I missing something completely here?10:19
acolesmahatic: which patch in that unit test in?10:20
mahaticacoles: https://review.openstack.org/#/c/20345410:20
acolesmahatic: k, looking10:21
mahaticokay10:21
*** jasondotstar has quit IRC10:22
acolesmahatic: yes. looks like that test is checking nothing blows up but the TODO needs to be implemented - maybe by having FakeApp stash the object body and then comparing that with the expected value.10:24
mahaticacoles: okay. So that goes into the unit test? and not probe? Should I be looking into adding that check in the unit test first?10:25
acolesmahatic: jrichli may think otherwise but yes implementing that check in unit tests first would seem a good thing to do and will be easier (i'd hope!) than a probe test10:28
acolesmahatic: but check with jrichli, she is more familiar with the code (i'm trying to catch up!)10:29
mahaticacoles: :) sure. I also think that would be a relatively easier step to begin with. But will check with her10:32
*** joeljwright has joined #openstack-swift10:34
*** ChanServ sets mode: +v joeljwright10:34
onovyzigo, hi, are you there?10:34
*** jasondotstar has joined #openstack-swift10:40
openstackgerritVictor Stinner proposed openstack/swift: Fix tox -e py34  https://review.openstack.org/19903410:42
haypoWARNING: Python 3 is coming! ------------^10:44
hayponotmyname: ok, i think that we know have all requirements to push this stuff ^^10:44
hayponotmyname: FYI i contributed to PyEClib to port it to Python 3, but also to fix their embedded version of liberasurecode to ease installation (fix how it's installed and detected in setup.py)10:45
hayponotmyname: they are running last tests on PyEClib 1.0.8 and they found a last bug on Centos (Tushar has a fix)10:46
hayposo the tempoary hack in tox.ini for PyEClib will go away quickly10:46
*** kota_ has quit IRC10:49
*** sakaYK has joined #openstack-swift10:54
*** sakaYK has quit IRC10:57
*** sakaYK has joined #openstack-swift10:58
*** jasondotstar has quit IRC11:01
*** jasondotstar has joined #openstack-swift11:03
*** akle|2 has joined #openstack-swift11:08
*** kei_yama has quit IRC11:13
*** ig0r__ has quit IRC11:15
*** akle has left #openstack-swift11:15
*** jasondotstar has quit IRC11:17
*** nakagawamsa has quit IRC11:17
*** sakaYK has quit IRC11:18
*** ig0r_ has joined #openstack-swift11:18
*** ig0r__ has joined #openstack-swift11:19
*** kei_yama has joined #openstack-swift11:19
*** akle|2 has quit IRC11:22
*** robefran has quit IRC11:29
onovythinking about one thing: why not put packagging information directly into official openstack-swift git repo? for example 'debian' directory11:29
*** sanchitmalhotra1 has quit IRC11:38
openstackgerritMerged openstack/swift: go: fix expecttransport memory usage  https://review.openstack.org/20627611:46
*** marzif_ has quit IRC12:05
*** joeljwright has quit IRC12:06
*** marcusvrn has joined #openstack-swift12:06
*** joeljwright has joined #openstack-swift12:21
*** ChanServ sets mode: +v joeljwright12:21
*** ig0r__ has quit IRC12:22
*** jasondotstar has joined #openstack-swift12:23
*** haigang has quit IRC12:25
*** silor has quit IRC12:27
*** ig0r__ has joined #openstack-swift12:27
*** thumpba has joined #openstack-swift12:28
openstackgerritMerged openstack/swift: go: make replicateDevice more defensive  https://review.openstack.org/20721712:29
*** thumpba has quit IRC12:33
*** bapalm has joined #openstack-swift12:42
*** marzif_ has joined #openstack-swift12:54
*** jasondotstar has quit IRC12:55
*** marzif_ has quit IRC12:56
*** marzif_ has joined #openstack-swift12:56
*** hrou has joined #openstack-swift12:57
*** jasondotstar has joined #openstack-swift12:59
*** km has quit IRC13:06
*** kei_yama has quit IRC13:09
*** jkugel1 has joined #openstack-swift13:16
*** marzif_ has quit IRC13:17
openstackgerritMerged openstack/swift: Refactor diskfile  https://review.openstack.org/19842913:22
openstackgerritMerged openstack/swift: Fix string formatting in dispersion cli command  https://review.openstack.org/20532613:23
*** annegentle has joined #openstack-swift13:33
*** ig0r__ has quit IRC13:34
*** ppai has quit IRC13:41
*** jrichli has joined #openstack-swift13:42
*** jlhinson has joined #openstack-swift13:48
haypoacoles, cschwede : hi. i'm going to holiday, i would be so happy to see this patch merged before leaving! https://review.openstack.org/#/c/199034/ "Fix tox -e py34"14:07
haypoyou can read the long history, or just notice the short "gate-swift-python34: SUCCESS" which is explicit enough, no?14:07
haypoi had to cut a whole forest of bugs to prepare this one :-p (port PyEClib, upgrade hacking to 0.10 and so fix pep8 issues, upgrade pbr to 1.3, add dnspython3 dependency, etc.)14:09
cschwedehaypo: looking14:09
cschwedehaypo: so thee hack in https://review.openstack.org/#/c/199034/11/tox.ini for pyeclib is still required?14:11
haypocschwede: it's still needed14:12
cschwedeok14:12
haypocschwede: i harass PyEClib developers everydays, and i sent many patches to fix their issues (not related to py3 anymore, more to packaging/deploy/install code)14:12
haypocschwede: i expect a release in a few days, maybe it will take 1 month14:13
haypocschwede: "We believe that the core functionality of the library is stable (i.e. erasure coding), but dealing with platform-specific stuff is always tricky and fickle." https://bitbucket.org/kmgreen2/pyeclib/issues/69/need-a-release-108-with-python-3-fixes14:14
mahaticjrichli: good morning!14:18
jrichlimahatic: good morning!14:19
jrichlimahatic: did you get past the pip issues?14:20
mahaticjrichli: did you happen to see the scrollback? I was wondering if I could start with14:20
jrichlimahatic: no, i just got in a bit ago, and was going through email.  scrollback next :-)14:20
mahaticsure! :)14:21
mahaticlemme wait then ;)14:21
haypocschwede: so do you have an opinion on this change? :)14:22
*** bapalm_ has joined #openstack-swift14:23
*** h00327910__ has joined #openstack-swift14:23
*** Fin1te has joined #openstack-swift14:24
cschwedehaypo: yes. just added my +214:25
cschwede:)14:25
*** annegentle has quit IRC14:26
*** bapalm has quit IRC14:26
haypocschwede: cool!14:27
haypocschwede: "For the skipped rfc822 import on Python 3, see for example the change Ia5ee2ead67e36e8c6416183667f64ae255887736 which replaces the rfc822 module with the email module." see my comment before yours ;)14:28
cschwedehaypo: i saw that :)14:28
jrichlimahatic: the unit test for the encrypter has a mocked encryption.  The TODO is to verify that that the encryption was envoked, but it is not integrated with the real encryption.14:28
jrichliacoles: was part of the intent of the probetest to test with the real encryption?14:28
*** tdasilva has quit IRC14:29
*** acoles has quit IRC14:29
mahaticjrichli: so that unit test is not supposed verify the real encryption? and any real encryption test goes into probe?14:30
*** moo41 has quit IRC14:31
jrichlimahatic: the first statement there is correct.  the other one - not sure what is typical.  I would have thought maybe a functest for that integration, but maybe its a probetest. waiting on acoles to express an opinion :-)14:31
mahaticjrichli: yes, okay :)14:32
jrichlimahatic: there is a unit test for the real crypto class itself.14:32
hroujrichli, mahatic - the difference between probe / func confused me as well ; )  From what I gather probe are more end to end usage scenario (maybe more like SVT testing, for those coming from a more traditional dev model), whereas FVT tends to target a very specific use case.  But I could be off base.14:35
jrichlihrou - my understanding is that the functests are testing a flow from an API call.  The other big thing is that the functests run with your SAIO.  So, if we add a PUT for testing encrypter,  it would be skipped if that is not in the pipeline.14:38
jrichlihrou - my understanding of the probetests is that they test more complicated things under the hood like replication.  they require more than the SAIO and will start and stop processes14:39
*** lcurtis has joined #openstack-swift14:39
jrichliso, encryption tests can fit in either case.  but I wanted acoles's opinion in this particular case because he had written a trello card for a probetest14:40
hroujrichli, thanks !  That makes sense, that's a good point re: SAIO by default it wouldn't run anything related to encryption (so I think there's a precedence there as I think there are other examples of middelware that aren't in the pipeline by default, but have func tests).14:41
mahaticjrichli: right, I'm looking at it (the crypto class). and thanks! ( for the info tests)14:44
mahaticinfo on tests*14:44
jrichlimahatic:  I haven't yet looked into the errors given on the existing probetests.  If you wanted to start digging into those, it would be a big help.  Record your findings on the etherpad.14:45
mahaticjrichli: okay. But now that you mention probe tests require more than an SAIO, it got me thinking ;)14:46
jrichlimahatic: I may have over-summarized there - you don't need to setup more than the SAIO for them to run.14:47
mahaticjrichli: okay! will look into them14:48
*** pgbridge has quit IRC14:49
*** breitz has quit IRC14:55
*** breitz has joined #openstack-swift14:56
*** annegentle has joined #openstack-swift14:57
*** chlong has quit IRC14:57
*** jasondotstar has quit IRC15:03
*** pgbridge has joined #openstack-swift15:07
*** thurloat is now known as thurloat_isgone15:08
*** chlong has joined #openstack-swift15:10
*** joeljwright has quit IRC15:18
*** joeljwright has joined #openstack-swift15:19
*** ChanServ sets mode: +v joeljwright15:19
*** joeljwright has quit IRC15:19
*** joeljwright has joined #openstack-swift15:19
*** ChanServ sets mode: +v joeljwright15:19
*** acoles_away has joined #openstack-swift15:26
*** jasondotstar has joined #openstack-swift15:26
*** tdasilva has joined #openstack-swift15:26
*** acoles_away is now known as acoles15:26
*** ChanServ sets mode: +v acoles15:26
*** zaitcev has joined #openstack-swift15:28
*** ChanServ sets mode: +v zaitcev15:28
notmynamegood morning15:28
tdasilvagood morning!15:30
mahaticgood morning!15:31
tdasilvanotmyname: still in Seattle?15:44
notmynameyup15:44
tdasilvacool! you got some nice pics from there!15:45
notmynamethe weather has been very clear and warm, so it's been easy :-)15:46
notmynameI think we may drive towards rainier tomorrow to see it up close15:46
tdasilvanice! enjoy!15:52
notmynameacoles: I think that patch isn't showing up (at the moment) because it got the +A vote, so it's not in the "needs another vote to land" section16:01
notmynamebut it is on the summary/overview dashboard (2nd link in the topic)16:01
notmynamein the "in the gate queue" section16:01
hayponotmyname: hey! nice pics on twitter yeah :)16:01
*** joeljwright has quit IRC16:01
notmynamebut I think I can tweak that rul a little16:01
*** rledisez has quit IRC16:01
notmynamehaypo: thanks16:01
hayponotmyname: just before i leave for ~1 month of holiday, can i get a +2a here? :-D https://review.openstack.org/#/c/199034/16:02
hayponotmyname: my patch to fix py34 gate16:02
haypowell, in fact, there is no major urgency, but someone may have to update my patch later16:04
zaitcevit's okay16:04
haypo(i hate timezones, i don't understand why the whole world don't live in my timezone :-D)16:04
hayponotmyname: do you live at seattle? or is it just for holiday?16:04
notmynamehaypo: when are you leaving?16:04
zaitcevMy god, that sed16:04
haypoi'm going to seattle next month for the KVM Forum with my Red Hat team16:05
haypozaitcev: \o/16:05
notmynamehaypo: no, I live in san francisco. I'm here for the week because my son is going to an FC Barca soccer camp16:05
hayponotmyname: ah, soccer, cool :) it's less popular in the USA than in europe16:05
haypopeople are crazy about soccer here. our "local" team in the the "OM", Olympique de Marseille16:06
haypo(not as popular as Barca? well, i don't know)16:06
*** jasondotstar has quit IRC16:06
haypozaitcev: the sed command should not stay for long. PyEClib 1.0.8 is "almost" there16:06
haypozaitcev: i sent my pull request in april. it takes times for some projects to merge pull requests and then cook a release. the sed is a compromise to not have to wait for an external dependency16:08
notmynameI'm working on (ie it's on my todo list) to get pyeclib more manageable for us16:08
hayponotmyname: they dropped jerasure & gf-complete tarballs, but there is still a liberasurecode tarball16:10
haypoi don't like embedding dependencies :-( but at least, i have to make it work correctly on my fedora 2216:10
haypooops16:10
haypoi helped* to make it16:10
hayposorry, i really have to go. see you maybe later. bye16:14
*** nadeem has joined #openstack-swift16:15
*** nadeem has quit IRC16:16
*** jlhinson_ has joined #openstack-swift16:16
*** nadeem has joined #openstack-swift16:16
*** jlhinson has quit IRC16:17
*** mahatic has quit IRC16:18
*** Fin1te has quit IRC16:20
*** ig0r__ has joined #openstack-swift16:21
notmynameon the review dashboard, do you want to see your own patches that only need one more +2 or do you want those filtered out? currently those are filtered out16:22
notmynameacoles: ^16:22
*** Fin1te has joined #openstack-swift16:28
*** jlhinson_ has quit IRC16:37
acolesnotmyname: no. having own patches filtered is cool. i have my own dashboard ;)16:38
notmyname:-)16:38
acolesnotmyname: re the 'missing' one - I hadn't +A'd that patch when i noticed it was missing on dashboard, it had +2 and -1 from jenkins16:38
notmynameah, ok16:38
acolesit no big deal16:38
notmynameI see that filter. I've taken it out16:38
acolesok cool16:39
acolesjrichli: yes re probe test16:39
acolesjrichli: but can't a unit test also test 'real' encryption? by grabbing the body output from the encrypter?16:40
acolesjrichli: mahatic was asking about the TODO in an encrypter unit test saying 'somehow verify encryption' or words to that effect16:40
*** robefran has joined #openstack-swift16:41
jrichliacoles: i was just thinking that maybe it should check that the crypto method had been called.  but that uses a mock for the crypto class since it has its own unit tests16:42
acolesjrichli: oic16:42
*** jistr has quit IRC16:42
tdasilvanotmyname: cool! didn't know your son was a soccer fan...did he get to hangout with barca players?16:42
tdasilvaor just coaches?16:43
acolesjrichli: imho the more coverage in unit tests the better since we don't run probe tests in a jenkins job16:43
acolesnotmyname: didn't barca play man utd in CA recently?16:43
jrichliacoles: ok.  but i think it will be good to keep the encrypter test ignorant of the crypto algorithm being used16:43
*** briancline has joined #openstack-swift16:43
notmynameacoles: yup, they did16:44
acolesjrichli: fair enough. so maybe another unit test that combines real encrypter and real crypto modules and verified the output?16:44
notmynametdasilva: this is part of the FCB escola thing (their training camp). there's a few coaches from spain here that are running the camp16:45
* portante brazil vs germany, 1-716:45
acolesnotmyname: i just realised than man utd are back in champions league this season and swift meetings now conflict with the end of the matches :/16:45
jrichliacoles: ok.  we will need to think of a way to verify the real crypto without being crytpo alg specific16:45
acolesportante: hi!16:45
portanteacoles: hello16:45
portantejust poking jabs at tdasilva here16:46
portante;)16:46
acolesportante: they don't play soccer in brazil do they ? ;)16:46
tdasilva:P16:46
portante=)16:46
tdasilvaportante: 516:46
notmynamejrichli: assertFalse(any(word in word_list for words in cyphertext.split()))16:46
*** marzif_ has joined #openstack-swift16:47
acolesjrichli: right16:48
jrichlinotmyname: sounds good!16:48
openstackgerritOpenStack Proposal Bot proposed openstack/python-swiftclient: Updated from global requirements  https://review.openstack.org/8925016:48
notmynamejrichli: tests pass! ship it!16:48
jrichlinotmyname: and if they dont pass, just remove the test ;-)16:49
jrichlibad test!16:49
*** amit213 has quit IRC16:50
*** amit213 has joined #openstack-swift16:50
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873616:50
openstackgerritjanonymous proposed openstack/swift: bin directory: :Replace python print operator with print function (pep H233, py33)  https://review.openstack.org/20660216:55
notmynameClint Dempsey just walked by the kids' soccer camp and they all got a picture with him16:58
acoleswho?16:59
notmyname(for those who don't follow soccer, he was the captain of the US team in the world cup)16:59
acoles;P16:59
notmyname"...the first American player to score a hat-trick in the English Premier League"16:59
notmynamehttps://en.wikipedia.org/wiki/Clint_Dempsey16:59
hroujrichli, yea validating something is encrypted is always fun, we had a similar issue with DB2 when we did our encryption work and used a similar solution to what notmyname described, but if you use a random IV and key .. ; )16:59
tdasilvaacoles: lol16:59
notmynamethe soccer camp is at the same place where the seattle sounders practice17:00
* tdasilva notmyname just keeps setting himself up for more acoles sarcasm17:00
acolestdasilva: notmyname nah my knife is actually sharpened for mattoliverau re cricket17:01
notmynameacoles: oh, that's the game that takes 4 days to play and has tea-time breaks, right?17:01
*** Fin1te has quit IRC17:01
acolesmattoliverau: btw pycharm tip - to update the pep8 version pycharm runs you need to copy pep8.py into the helpers dir17:02
acolesmattoliverau: at least that wfm on 3.4.1 community ed. ymmv17:02
acolesmattoliverau: thats if you want pycharm to moan the same way that tox -e pep8 does with the newer hacking version17:03
acolesnotmyname: thats the game that has 5.5 ounces of hard ball hit you at >80mph if you dither17:03
acolesbut the teas are good!17:03
acolesnotmyname: https://s1.yimg.com/bt/api/res/1.2/kpfrfPafsRY2Pkc12ZiTRg--/YXBwaWQ9eW5ld3M7Zmk9aW5zZXQ7aD0zNTM7aWw9cGxhbmU7cT03NTt3PTYzMA--/http://media.zenfs.com/en_us/News/Reuters/2014-08-09T130609Z_1243708996_GM1EA891MIJ01_RTRMADP_3_CRICKET-INDIA.JPG17:05
notmynameacoles: https://www.youtube.com/watch?v=dEH4ahCCrJo17:06
openstackgerritAlistair Coles proposed openstack/swift: Add POST capability to ssync for .meta files  https://review.openstack.org/13849817:06
tdasilvaacoles: acoles you just reminded me of this: https://www.youtube.com/watch?v=fzqsqBXMy7g17:07
acoleslol17:07
acolestdasilva: but he's wearing a glove!17:08
tdasilvaacoles: would you do that wearing a glove?17:08
*** chlong has quit IRC17:09
*** annegentle is now known as theannegentle17:09
*** jrichli has quit IRC17:09
acolesclayg: i finally got patch 138498 updated & ready for review (ssync/post)17:10
patchbotacoles: https://review.openstack.org/#/c/138498/17:10
*** jrichli has joined #openstack-swift17:11
acolestdasilva: https://www.youtube.com/watch?v=E_ANBewmvhA17:12
tdasilvaacoles: wow! touché!17:13
*** SkyRocknRoll has joined #openstack-swift17:17
*** theannegentle has quit IRC17:19
*** theannegentle has joined #openstack-swift17:20
*** acoles is now known as acoles_away17:24
*** ig0r__ has quit IRC17:24
openstackgerritThiago da Silva proposed openstack/swift: versioned writes middleware  https://review.openstack.org/13434717:25
*** jasondotstar has joined #openstack-swift17:25
tdasilvaclayg: around? when you have a chance please checkout latest comments on patch 134347. I think upgrade path got a bit more complicated17:26
patchbottdasilva: https://review.openstack.org/#/c/134347/17:26
*** ChanServ changes topic to "Review Dashboard: https://goo.gl/eqeGwE | Summary Dashboard: https://goo.gl/jL0byl | Hackathon topics: https://etherpad.openstack.org/p/swift-midcycle-aug-2015 | https://etherpad.openstack.org/p/swift_encryption_issues | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/"17:27
openstackgerritMerged openstack/python-swiftclient: Test auth params together with --help option.  https://review.openstack.org/20732217:33
*** theannegentle has quit IRC17:45
*** aix has quit IRC17:54
*** robefran has quit IRC18:03
*** theannegentle has joined #openstack-swift18:11
*** breitz has quit IRC18:12
*** Fin1te has joined #openstack-swift18:14
*** jordanP has quit IRC18:21
*** geaaru has quit IRC18:22
*** marzif_ has quit IRC18:25
clayghey-oh18:49
claygtdasilva: no, i'll only look at if you say it got *better*18:49
* haypo is back18:50
haypotimburke: for pbr, it's possible to specify the minimum pbr version in setup.py18:50
timburkehaypo: yes. we don't currently, but if we're going to use env markers, we should18:51
haypotimburke: but i don't know what's the best option18:51
haypolifeless is our local pbr expert18:51
* haypo invokes lifeless 18:51
haypolifeless lifeless lifeless18:51
zaitcevI run git-review on Python 3 and man, the side effects are hilarious. Can't wait to see Swift doing that.18:53
haypozaitcev: what are the side effects?18:54
zaitcevhaypo: git-review -v  tracebacks immediately, because py3 has different exception variable scope18:55
zaitcevUnboundLocalError: local variable 'no_git_dir' referenced before assignment18:55
haypozaitcev: ah? i'm using git review regulary in .tox/py34 venv18:55
zaitcevhaypo: git-review -l  prints this: b'37781'            b'master'  b'Cache check mount calls'18:55
haypoi mean, i'm running git review everydays, multiple times, in various projects18:56
haypoand i never got your error18:56
zaitcevof course18:56
haypozaitcev: i have git-review 1.25.018:56
zaitcevonly a complete n00b would run it without arguments and outside of git repository18:56
haypoit's installed for python 2, not python 318:57
zaitcevas long as you either run it in a repo, or provide it meaningful arguments, it will not traceback18:57
zaitcevhaypo: anyway, see https://review.openstack.org/20037118:57
*** Fin1te has quit IRC18:57
haypo"or provide it meaningful arguments" which command makes failing git review?18:58
haypozaitcev: but yeah, it would be nice to have git-review working on python 3 ;)18:59
haypozaitcev: py26 & py27 failed on your patch :p)19:00
zaitcevhaypo: They failed because of mock 1.119:00
haypozaitcev: 1.1? mock 1.3 is out!19:01
zaitcevhaypo: well, the py26 did. The py27 failed because of something called "Gerrit WAR".19:01
zaitcevhaypo: well then, let's have infra guys install it and then my patch may go in19:01
haypozaitcev: i'm not sure that mock is in the part. it's installed by tox using requirements, no?19:02
zaitcevhaypo: I linked this when we discussed it previously, but https://bugs.launchpad.net/openstack-gate/+bug/147328619:04
openstackLaunchpad bug 1473286 in OpenStack-Gate "mock fails with SyntaxError in py26" [Undecided,Invalid]19:04
zaitcevhaypo: And either your or lifeless explained that they fixed it up in 1.419:05
zaitcevbecause someone volunteered to do it19:05
haypozaitcev: i didn't work on mock19:06
haypobut i helpd to fix swift (and some oslo projects) for mock 1.119:06
haypobecause mock become more strict and helped to find bugs in tets19:07
haypotests*19:07
zaitcevI know. But that only worked because we do not support py26 anymore. 1.1 being dead in py26 is not something you can fix in a project code. The only thing a project can do is this: https://review.openstack.org/20086019:08
openstackgerritVictor Stinner proposed openstack/swift: Fix tox -e py34  https://review.openstack.org/19903419:11
haypotimburke, zaitcev : ^^ i ran "tox -e update ../swift" in requirements. it added the requirements pbr>=1.3 in setup.py19:11
haypotimburke: is it better?19:11
*** jasondotstar has quit IRC19:12
*** ig0r__ has joined #openstack-swift19:12
*** SkyRocknRoll has quit IRC19:14
*** jasondotstar has joined #openstack-swift19:19
timburkehaypo: hmm. maybe that wasn't sufficient. i expected http://paste.openstack.org/show/406627/ to Just Work, but despite installing a newer pbr, it doesn't seem to use it19:21
openstackgerritMichael Barton proposed openstack/swift: go: fix some panics in unpickle  https://review.openstack.org/20803319:22
*** jasondotstar has quit IRC19:25
haypotimburke: i heard that this is a ckicken-egg issue19:25
*** ig0r_ has quit IRC19:26
haypotimburke: but "tox -r -e py34" works for me, whats the use case?19:26
timburkehaypo: i believe it. but i still want a way to upgrade from a previous install19:26
haypotimburke: the way is simple: tox -r -e py3419:27
haypotimburke: or rm -rf .tox19:27
haypotimburke: does it work if you upgrade pbr to 1.3? pip install -U 'pbr>=1.3'19:28
haypotimburke: you should only have to do this once19:28
haypobut yeah, you dev environment may need some manual steps if you don't want to use tox -r19:29
tdasilvazaitcev: hi, have you seen this error before: http://ur1.ca/nbh2519:29
haypotdasilva: do you have Jerasure installed?19:29
timburkehaypo: no, i mean given an old deployment of swift, i should be able to install the new version. requiring users to upgrade pbr out-of-band feels busted to me19:29
haypotdasilva: oops, you don't have Jerasure :) you should install it19:30
timburkehaypo: it's not tox that i'm worried about. tox can sort itself out19:30
haypotimburke: Red Hat packages don't use pbr :)19:30
zaitcevtdasilva: Swift in Rawhide is super fragile and I think I made it build just once. Tons of issues like that.19:30
haypotimburke: who deploy using raw source code?19:30
haypoin a package, you have a full control on requirements19:31
zaitcevhaypo: Agile DevOps ninja rockstars all do.19:31
haypobetter than our fragile setup.py, no?19:31
timburkehaypo: fair enough. maybe it's my thinking that's busted19:31
haypozaitcev: lol19:31
haypoi heard that people don't really upgrade clouds19:31
haypoyou spawn a new one and migrate VMs19:32
haypoi was surprised, but well, it makes sense :)19:32
lifelesszaitcev: but thats wrong19:32
hayposorry, i don't have any experience in deployment. or well, small experience19:32
*** openstack has joined #openstack-swift19:35
*** ig0r__ has quit IRC19:35
lifelesstimburke: so that pastebin isn't pbr19:35
lifelesstimburke: http://paste.openstack.org/show/406627/ - its setuptools being too old19:36
lifelesstimburke: you can add a setup_requires versioned dep on setuptools>=17.119:36
lifelesstimburke: but it won't make it automatically work, because setuptools doesn't handle upgrades to itself19:37
haypolifeless: ah yes, that's the chicken-egg issue19:37
lifelessI'd like to do a lot better19:38
lifelessand I have some prototype code to do so, but its still working through the experimental stage19:38
*** NM1 has joined #openstack-swift19:39
haypolifeless: what's the impact for swift developers and swift deployers of upgrading pbr from 0.11 to 1.3?19:39
*** ig0r_ has joined #openstack-swift19:39
haypodevelopers should recreate their virtualenv, that's all, no?19:40
timburkelifeless: ah, ok. so install setuptools>=17.1 and pbr<1.0, *then* running setup.py should use the deps it's installed and work?19:42
claygtimburke: does that work?19:44
*** ig0r_ has quit IRC19:46
openstackgerritMichael Barton proposed openstack/swift: go: fix timing issue in ring test  https://review.openstack.org/20804119:47
timburkeclayg: nope :( http://paste.openstack.org/show/406628/19:48
timburkelifeless: no, setuptools is apparently up-to-date (18.0.1)19:49
lifelesstimburke: so yeah, thats likely just another symptom of the inability of setuptools to 'fix' unsatisfied setup_requires :(19:51
lifelesstimburke: but - what is your setup.py here?19:51
lifelesstimburke: cn you pastebinit, or is there a review I can pull down?19:51
timburkelifeless: https://review.openstack.org/#/c/199034/12/setup.py - basically, add a min pbr version to setup_requires19:52
ccavanna_Hi. I have a question about git. Hoping anyone can help me. I have a local branch I've been working on for a few days now (and commited changes).19:52
ccavanna_I am addressing some of the reviews at the moment.19:53
ccavanna_As part of that, I decided to run: git pull origin master19:53
ccavanna_(to get everything up to date)19:53
haypolifeless: ah, there is a question for you (me?) on https://review.openstack.org/#/c/199034/12/setup.py19:53
ccavanna_I changed my file (just a doc)19:53
haypozaitcev, lifeless : i don't know how pbr requirements should be specified, so i copied what was done in nova. in nova, pbr>=1.3 is in setup.py and requirements.txt19:54
haypoi guess that it's required because of the chicken-egg issue19:54
zaitcevhaypo: I see. Okay19:54
claygtimburke: but I don't understand - can you just pip install --upgrade setuptools pbr and then it works?19:54
openstackgerritMinwoo Bae proposed openstack/swift: Error handling of DiskFileNoSpace()  https://review.openstack.org/19890919:57
lifelesstimburke: oh btw - does pip install . do any better?20:01
haypozaitcev, timburke, clayg : i'm sorry, i didn' follow the discussion closely, i'm doing something else at the same time. did you fix you issues with my "tox -e py34" patch? https://review.openstack.org/#/c/199034/20:03
clayglol - timburke oh yeah i forgot that part - you can't use python setup.py install anymore - it's all pip install .20:04
claygoh my favorite pip install -e .20:05
claygi can't even type python setup.py develop anymore - but I guess my build scripts are going to be pissed if I can't get python setup.py sdist to work20:05
claygI don't know how to spell sdist in pip - but i bet it's possible20:05
lifelesspython setup.py sdist :)20:09
lifelesspip knows how to wheel20:09
lifelessand how to install from an sdist20:10
lifelessbut not how to make an sdist20:10
lifelessonce we have declarative deps, it may make sense to make an sdist thing20:10
lifelessor at least an install-my-build-deps20:10
lifelessthing20:10
haypoclayg: i never call setup.py myself, i always rely on tox. you don't use tox?20:11
claygto install the code in my development environment and virtualenvs?  for swift or like other python things I work on?20:11
claygI used tox; but using it doesn't make me not run things like 'pythno setup.py sdist' or 'pip install -e .' sometimes20:11
haypoclayg: you need maybe to write a "pbr survivor guide" and share it with other developers :)20:15
*** zhill has joined #openstack-swift20:16
redboI'm glad it add so much value20:16
redboadd*20:16
haypoi still don't understand why we use pbr :)20:16
redboack what's wrong with my s key20:16
haypobut i'm not interested to work on this :-D20:16
redbonobody knows why we need it, but we're not allowed to get rid of it20:17
haypothe nice thing is setup.cfg20:17
*** robefran has joined #openstack-swift20:17
*** robefran has quit IRC20:17
hayporedbo: not allowed?20:17
redboI know!20:18
haypofor my understanding, there is is not good packaging tool, pbr is maybe the least worst tool :)20:18
hayporeminder: right now, i only care of python3 :)20:19
redbowhat does it really add?  git versioning is about it as far as I can tell.20:19
hayporedbo: but lifeless can maybe reply to your questions. he knows well pip, pbr, setuptools, tox, etc.20:19
hayporedbo: it adds support for setup.cfg, packaging stuff written in a configure file, instead of a python program20:20
claygredbo: pbr was trying to work around shit that setuptools-0.6c did that was stupid I'm pretty sure20:20
hayporedbo: it's better to use a configuration file than executing python code20:20
redbodoesn't setup.cfg come from distuils?20:20
redbodistutils20:20
hayporedbo: no20:20
hayporedbo: it comes from distutils2 which died in pain20:20
lifelesssetup.cfg is used by setuptools, but only minimally20:20
haypo(and two other developers :-D)20:20
lifelessits also used by wheel20:20
lifelessand d220:21
lifelessand pbr20:21
lifeless=> its a common config file20:21
openstackgerritCarlos Cavanna proposed openstack/swift: Improving statistics sent to Graphite.  https://review.openstack.org/20265720:21
redbookay20:22
claygredbo: lol!20:22
clayg^ AND THAT IS HOW THE MAGIC HAPPENS FOLKS20:22
redboI still don't think that's worth all the pain its put us through20:22
lifelessredbo: pbr gets us versions in docs; declarative dependencies that can be machine introspected; git versioning; support for PEP-426 markers in fields that setuptools doesn't support yet20:22
hayporedbo: you're free to propose something else20:22
redboI've tried20:22
clayglol20:23
hayporedbo: but i don't think that you want to know the dark side of python packaging20:23
hayporedbo: most developers who tried lost their mind, we lost them forever20:23
haypothey are now ermits in high mountains20:23
clayghaypo: that's also true!20:23
haypohum. maybe i exagerate sometimes20:24
redboWell you say that, but there's like 20,000 python libraries that don't use pbr and manage to get installed20:24
claygno no no - it's all true20:24
haypoclayg: haha20:24
haypoclayg: tarek ziade and eric araujo wrote distutils2. both left the python project20:24
haypoeric also left france20:24
claygyeah yeah tarek - that's the guy20:25
haypoi didn't see tarek since more than 2 years20:25
haypoi'm not sure that he still exists!20:25
zaitcevI went to Monty's talk once like 3 summits ago in Atlanta or Sun Antonio and basically he listed some issues with packaging that prompted PBR. But none of them applied to Fedora, so I forgot all of it.20:25
haypozaitcev :)20:25
claygsorry this is *so* therapeutic for a Friday20:26
haypoah, tarek is still alive :) http://blog.ziade.org/20:26
lifelessredbo: theres more like 10 projects with the developer base of swift20:26
lifelessredbo: and maybe 2 with the developer base of openstack as a whole20:26
haypoi just didn't met him since many years, he doesn't go to pycon fr anymore20:26
redboI'm sure the django project is about to fall apart with the overhead of maintaining their setup.py file20:27
hayporedbo: for me, using a configuration file (setup.cfg) is more secure, more portable, and may be faster20:28
swifterdarrellredbo: I think they still put the version in a file.  like Neanderthals20:28
claygoh god - no it's too much - please stop20:28
claygROFL20:28
hayporedbo: i also like wheel packages. pip 7 helped a lot, it now builds automatically wheel transparently20:29
redboOkay, let's say we need *something* there.  Does it have to be something that causes endless pain just to translate .cfg files to setup() arguments?20:31
hayporedbo: ok, to discuss more seriouly, _all_ packaging tools are still based on distutils. pip is based on setuptools which is based on distutils20:32
claygredbo: I think you're transfering some of pain from new features in setuptools and pip to pbr - it *started* with pbr - but now thye're fixing *everything* - it's a new world!20:33
hayporedbo: it became impossible to fix the underlying layer (distutils) because it would break too much things20:33
claygI remember back when python developers used to be into "move slow and just work"20:33
hayposo we are f***ed20:33
claygmove fast and break things20:33
* clayg pops off another six shooter20:33
hayposaid differently, nobody wants to start a new fresh "distutils" project from scratch20:33
redboclayg: that may be true.  Any time running seutp.py completely screws up my machine, I kind of blame pbr.20:34
claygredbo: yeah :) it was downhill from dist is not a Distribution20:35
haypoclayg: it's impressive how much bugs and new cool features are now added to pip20:35
haypooops20:35
haypoi mean, bugs are fixed, new features are added, sorry :-D20:35
hayporedbo: pbr is a small piece. there are pip, setuptools and distutils behind20:36
zaitcevSubject: Re: Transitioning Ceph from Autotools to CMake20:36
zaitcevThere is no escape20:36
redboclayg: Oh man, it still hasn't really even been all that long since I saw dist is not a Distribution.  Then I google for it, and I just see your post to distutils mailing list or whatever.20:36
haypofrom my point of view, it's only become better20:36
clayghaypo: every day is a little better than the next20:37
lifelessredbo: running setup.py should never screw up your machine20:37
lifelessredbo: pbr or not20:37
redboI agree20:38
lifelessredbo: but sadly, easy_install pretty much guarantees that running setup.py directly will do so20:38
lifeless:/20:38
openstackgerritDavid Goetz proposed openstack/swift: Speed up reaper for a big account delete and some better error handling  https://review.openstack.org/20805720:38
*** bapalm_ has quit IRC20:38
redboI don't know, lots of python stuff, I run setup.py and nothing too bad happens.20:39
zaitcevI simply never run anything with sudo (don't even have sudo installed), and I never run setup.py as root.20:40
*** michaeljk has joined #openstack-swift20:40
*** zhill has quit IRC20:41
claygzaitcev: is stright up baller20:44
haypotwo years ago, i dind't understand the purpose of virtualenv20:45
hayponow i'm only using it to develop20:45
haypoit like being install to install whatever, modify code of libraries, hack pip, hack everyhing20:45
haypoand then simply destroy it to rebuild a fresh virtualenv20:46
*** aix has joined #openstack-swift20:47
redboBut it's kind of like complaining about the weather.20:47
claygredbo: well if the weather is shitty all the time20:48
redboI know, it's a perfect metaphor20:48
haypodon't forgot to review my patch :) python3 is not the future, but the present!20:49
clayghrmm.... it seems like there is an alternative future we could go with20:49
claygif the weather is so crappy then why don't you just move?20:50
lifelessfor instance.. https://bitbucket.org/kmgreen2/pyeclib/issues/62/pip-install-uninstall-install-errors-with20:50
claygtoo much fun20:50
redboI did!20:50
lifelessthats the sort of thing setuptools/pip/pbr get blamed for20:50
zaitcev"go with" rimshot20:50
lifelessbut nothing to do with them20:50
lifeless(except insofar that they don't force real isolation etc... postels tarpit)20:50
redboas soon as there's a gbr, I'll just start over again in rust or something.20:52
* clayg snickers20:52
haypolifeless: PyEClib setup.py is a little bit strange. it installs manually a C library (liberasurecode) using ./configure --prefix=/usr/local && make && make install20:53
claygacoles_away: thanks for doing the v3 dispersion change20:53
haypolifeless: did you notice my comment? it should now be fixed20:53
openstackgerritMichael Barton proposed openstack/swift: go: fix timing issue in ring test  https://review.openstack.org/20804120:54
lifelesshaypo: perhaps. easy enough to test20:54
haypolifeless: I added /usr/include/liberasurecode to paths used to locate C headers, and I modified --prefix=/usr/local with --prefix=/usr20:55
haypolifeless: (and embedded liberasurecode was upgrade from 1.0.7 to 1.0.8)20:55
lifelesshaypo: its not clear to me why that would fix it20:56
lifelesshaypo: anyhow - as i said, easy enough to test20:56
haypolifeless: it worked for me :)20:57
haypolifeless: the error is "erasurecode.h: No such file or directory". liberasurecode installs its headers in /usr/include/liberasurecode, but /usr/include/liberasurecode was not in "include dirs"20:59
*** jrichli has quit IRC21:00
claygtdasilva: so i'm not 100% on what I need to look at on versioned writes - upgrades are working according to your fpaste yeah?21:01
haypolifeless: but i'm not sure that i understood your usecase :)21:02
claygtdasilva: oic - removing the container option blocks old containers - bah21:02
lifelesshaypo: I need to go get food n stuff21:04
*** michaeljk has left #openstack-swift21:06
haypoi have to go away from computers :)21:07
haypobye21:08
*** haypo has left #openstack-swift21:08
*** changbl has quit IRC21:09
*** NM1 has quit IRC21:10
claygdfg_: does shard mean - each replica of the account will process the containers modulo?21:23
dfg_clayg: ya. shard is a word right?21:25
dfg_its basically my solution to everything. that and ratelimiting21:27
*** bapalm_ has joined #openstack-swift21:31
hrouHey All, after picking up haypo's change to tox.ini (essentially changed it from a white list to a black list) anyone else seeing a bunch of failures in PEP8 with 'H402  one line docstring needs punctuation'  ?21:34
*** DericHorn-HP has joined #openstack-swift21:41
*** mahatic has joined #openstack-swift21:42
hrouWe didn't use to check H402 so that would make sense, but I wagered someone would have ran into this already by now so maybe there's more to it : ) i.e. something I'm missing.21:44
*** mahatic has quit IRC21:44
lifelesshrou: did you also update your hacking ?21:51
lifelesshrou: since that check came and went IIRC21:51
claygdfg_: yeah totally nice work!21:52
claygdfg_: I looked it over and had a few first impressions - but I haven't reviewed it yet because I have another stupid change checked out21:52
claygdfg_: do you want me to push my comments - or do you want me to wait till I run it?21:52
claygdfg_: it'll probably just be a little bit21:53
hroulifeless, ah that's probably it I'm on 0.8.121:55
*** bapalm_ has quit IRC21:55
clayghrou: sorry bro - I think pip install --upgrade -r test-requirements.txt should get you goin?21:55
hrouclayg, haypo, thanks a bunch, indeed it does !  Given its in test-requirments (i.e. the newer hacking version) that did the trick, err silly me21:57
claygyeah that + rebase on master should get you going - weee fun times!21:58
claygmove fast and break things!21:59
claygyee haw!21:59
*** jkugel1 has left #openstack-swift22:03
claygso cschwede's comment about "this test should have to change" got me thinking on patch 20419622:17
patchbotclayg: https://review.openstack.org/#/c/204196/22:17
hrouAn FYI for all ( I should add this to that new doc btw ; ))  ... you want to also do:  "tox -r"  to recreate the VM22:17
claygthe only reason that test has to change is because it bypasses some swob machinery - if it just called it like anyone else swob would cook up a request if it didn't get one -> https://gist.github.com/clayg/1e8cbd0b90eb66eb3ae0#file-request-is-should-patch-L2622:18
claygbut then I was like - oh hrmm... swob is cooking up a request when you call it if you don't get one - i bet we do that all the time! -> https://gist.github.com/clayg/1e8cbd0b90eb66eb3ae0#file-request-is-should-patch-L1022:18
claygthen I was like "we should audit everywhere we return a Request and..." and basicalloy passed out from exahustion22:19
claygcan ask the author of this stupid^Wnew feature to do this work - or do we just have to get ready for all the "this *one* swift error *doesn't* return json when I want it to!" bugs :'(22:20
claygwell - idk what to tell him - but I don't want to +2 it anymore either :'(22:21
claygi'll do a +1 and see what happens22:21
*** lcurtis has quit IRC22:25
*** nadeem has quit IRC22:25
*** hrou has quit IRC22:33
*** NM1 has joined #openstack-swift22:36
*** minwoob_ has quit IRC22:37
*** theannegentle has quit IRC22:48
*** NM1 has quit IRC22:50
claygwhat's the python for taking a list and grouping it into tuples?22:56
claygi know there's an itertools that does this...22:57
claygit's like the oppose of chain22:57
claygpairwise recipe I think!22:59
StevenKzip() ?23:01
*** theannegentle has joined #openstack-swift23:01
StevenKNope, that's for two lists. :-/23:02
claygbah - no pair wise isn't it23:04
claygit's like a zip of an islice after you tee it23:04
*** theannegentle has quit IRC23:06
claygzip(*[iter(info_lines)] * 2)23:09
claygyeah so making the list that all have the same reference to the iter for zip is pretty cleaver23:10
claygpoints go to torgomatic for pointing me at grouper23:10
claygdon't forget to call iter tho - that trick doesn't work with lists23:10
clayg(obvs.)23:10
*** DericHorn-HP has quit IRC23:22
*** DericHorn-HP has joined #openstack-swift23:25
*** jrichli has joined #openstack-swift23:42
*** hrou has joined #openstack-swift23:43
*** marcusvrn has quit IRC23:43
openstackgerritMerged openstack/swift: Fix regression in WSGI server SIGHUP behavior  https://review.openstack.org/20763723:50
*** jrichli_ has joined #openstack-swift23:52
*** jrichli has quit IRC23:53
*** theannegentle has joined #openstack-swift23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!