| *** pgbridge has quit IRC | 00:02 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-swiftclient: Updated from global requirements https://review.openstack.org/89250 | 00:05 |
|---|---|---|
| *** Jeffrey4l_ has joined #openstack-swift | 00:10 | |
| openstackgerrit | Merged openstack/swift: go: adding a x-trans-id to repconn along with the sender device id https://review.openstack.org/313167 | 00:13 |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Skip SLO-reconciling probe test when SLO is off https://review.openstack.org/312698 | 00:13 |
| notmyname | openstack clippy: "I see you've sent 4 emails to the ML today, and you started a thread that has over 75 replies. Can I order you a beer?" | 00:14 |
| *** suyash has quit IRC | 00:20 | |
| *** diogogmt has joined #openstack-swift | 00:25 | |
| *** lyrrad has quit IRC | 00:30 | |
| mattoliverau | notmyname: +1 I think you deserve a beer | 00:30 |
| mahatic | good morning | 00:31 |
| mattoliverau | mahatic: morning | 00:32 |
| mahatic | mattoliverau: o/ | 00:34 |
| kota_ | mahatic: morning | 00:37 |
| kota_ | mattovierau: I think notmyname can get a bunch of glasses of beer from all swifters | 00:40 |
| *** portante has joined #openstack-swift | 00:44 | |
| *** ndk has joined #openstack-swift | 00:45 | |
| *** dmellado has joined #openstack-swift | 00:49 | |
| *** tqtran has quit IRC | 00:54 | |
| *** sgundur- has left #openstack-swift | 00:57 | |
| tdasilva | notmyname: so far the best part of that long thread was: "I don't want to start a religious war...but...If you think Perl is "nice" or "easy" you better get you head checked....If you think Perl is "nice" or "easy" you better get you head checked....C++ is not worth mentioning as a language....don't force OpenStack to accept code written in $COOL_LANGUAGE because we don't all want to have to learn that language in add | 01:13 |
| tdasilva | ition to Python" | 01:13 |
| *** klrmn has quit IRC | 01:15 | |
| jrichli | notmyname: What are you doing with that VM that you end up with 'Too many open files' ? | 01:15 |
| jrichli | re: supporting Go mail thread : all I can say is, wow. | 01:40 |
| *** diogogmt has quit IRC | 01:44 | |
| *** arch-nemesis has quit IRC | 01:47 | |
| *** tqtran has joined #openstack-swift | 01:51 | |
| openstackgerrit | Merged openstack/swift: SwiftLogFormatter will log transaction IDs on INFO level https://review.openstack.org/309524 | 01:53 |
| jrichli | notmyname: it looks like your test skipped the container-sync tests | 01:55 |
| *** tqtran has quit IRC | 01:57 | |
| *** klamath has quit IRC | 01:59 | |
| *** klamath has joined #openstack-swift | 02:00 | |
| *** ndk has quit IRC | 02:00 | |
| *** ndk has joined #openstack-swift | 02:02 | |
| *** klamath has quit IRC | 02:03 | |
| *** klamath has joined #openstack-swift | 02:03 | |
| notmyname | yeah, that ML thread is fun, right? :-) | 02:06 |
| lifeless | notmyname: 'fun'. I'm not sure I speak the same language :) | 02:14 |
| lifeless | notmyname: FWIW I'm not sure we *should* try to limit languages in the big tent; thus the conditional on my reply | 02:14 |
| *** klrmn has joined #openstack-swift | 02:21 | |
| *** furlongm has quit IRC | 02:37 | |
| *** klamath has quit IRC | 03:00 | |
| janonymous | Please review patch 314100 , i | 03:06 |
| patchbot | janonymous: https://review.openstack.org/#/c/314100/ - openstack-infra/project-config - Add non-voting gate-swift-tox-bandit job | 03:06 |
| janonymous | It would require +1 from swift cores to move forward as commented by Andreas | 03:06 |
| notmyname | janonymous: the entirety of what I can find out about that is "Bandit provides a framework for performing security analysis of Python source code" | 03:07 |
| notmyname | what does this test actually test for? what is it preventing, and what is it showing in the results? | 03:08 |
| janonymous | notmyname: yes it is a static code analyser for security checks | 03:08 |
| notmyname | https://wiki.openstack.org/wiki/Security/Projects/Bandit | 03:09 |
| janonymous | notmyname: it tests against using of possible security issues which could be configured as per projects needs, as some checks might not be seen as security issue in some projs | 03:10 |
| notmyname | janonymous: do you have an example of output run against swift? | 03:10 |
| janonymous | you can run tox -e bandit on swift code | 03:11 |
| janonymous | by changing the values of n to different values will show possible medium and low severity issues , http://git.openstack.org/cgit/openstack/swift/tree/tox.ini#n64 | 03:13 |
| *** mingdang1 has joined #openstack-swift | 03:16 | |
| janonymous | just for example try: replacing s/-p gate/-lll | 03:16 |
| notmyname | bandit: commands succeeded | 03:16 |
| notmyname | $ tox -e bandit 2>&1 | grep 'Bandit internal error' | wc -l | 03:17 |
| notmyname | 13826 | 03:17 |
| notmyname | somehow, I don't believe the first one | 03:17 |
| *** Jeffrey4l_ has quit IRC | 03:18 | |
| janonymous | m sry try from these: bandit -c bandit.yaml -r swift bin -n 5 -lll / bandit -c bandit.yaml -r swift bin -n 5 -ll / bandit -c bandit.yaml -r swift bin -n 5 -l | 03:19 |
| *** Jeffrey4l_ has joined #openstack-swift | 03:19 | |
| *** sgundur- has joined #openstack-swift | 03:20 | |
| *** sgundur- has left #openstack-swift | 03:20 | |
| notmyname | janonymous: I'm not opposed to running more tests in the gate | 03:21 |
| notmyname | however, more tests does not always mean better | 03:21 |
| notmyname | I don't want to add a test that becomes the new longest-running one | 03:22 |
| janonymous | notmyname: yes, you are right | 03:22 |
| notmyname | I don't want a test that randomly fails | 03:22 |
| janonymous | notmyname: but what if we keep this non-voting | 03:22 |
| janonymous | notmyname: it's a good practice though not a necessity :) | 03:23 |
| notmyname | I want tests that give me clear results with a path to fix them and a way to reproduce tehm locally | 03:23 |
| janonymous | notmyname: bandit gives that options.. | 03:25 |
| notmyname | janonymous: what counts as passing of failing for bandit? right now I run these commands locally and get some interesting output and yet it says that everything passed | 03:26 |
| *** _JZ_ has joined #openstack-swift | 03:26 | |
| notmyname | janonymous: also what about andreas's question about adding it to tox.ini? | 03:27 |
| janonymous | yes because issues with low/medium priority will not fail invocation | 03:27 |
| janonymous | notmyname: since it is a non-voting job now, it might be not be needed at this point i think | 03:28 |
| notmyname | any ideas why this fails when runnign under tox but works when running locally? | 03:29 |
| janonymous | Some options that were tried are in this commit also ,for reference: https://review.openstack.org/#/c/196395/ | 03:29 |
| patchbot | janonymous: patch 196395 - swift - Adding bandit for security static analysis testing... (MERGED) | 03:29 |
| *** links has joined #openstack-swift | 03:29 | |
| janonymous | notmyname: can you give me a case.. | 03:30 |
| notmyname | the 13.8k lines of errors I got when running it with `tox -e bandit` | 03:31 |
| notmyname | actually seems like the same error for just about every line checked: https://gist.github.com/notmyname/7b75487bf14de7e86fe30c70a65741a8 | 03:32 |
| janonymous | somthing is wrong with env, bcs i get this as output: https://gist.github.com/codevulture/190ecc35071e69f809008227b7c733f4 | 03:36 |
| janonymous | i am checking.. | 03:36 |
| *** silor has joined #openstack-swift | 03:40 | |
| jrichli | notmyname sgundur- running probetests against different versions has shown that it was patch 270961 that broke the container_sync probetests | 03:40 |
| patchbot | jrichli: https://review.openstack.org/#/c/270961/ - swift - Container-Sync to perform HEAD before PUT object o... (MERGED) | 03:40 |
| jrichli | although the vm-saio-probe passed at the time. And it did not skip the container-sync tests. but if I run with the change just before that one, they pass in my VM. With that change, the 3 fail. | 03:51 |
| janonymous | notmyname: can you try out with bandit==0.17.3 version, maybe same as https://bugs.launchpad.net/bandit/+bug/1546777 :( | 03:53 |
| openstack | Launchpad bug 1546777 in Bandit "internal error running: blacklist" [High,Fix released] - Assigned to Tim Kelsey (tim-kelsey) | 03:53 |
| *** tqtran has joined #openstack-swift | 03:53 | |
| notmyname | janonymous: still errors with that | 03:54 |
| janonymous | notmyname: same ones ? | 03:56 |
| *** tqtran has quit IRC | 03:57 | |
| zaitcev | "Secure Web Internet File Transmission Service (SWIFT) Transmitter User Guide. Overview;" WTF | 03:58 |
| notmyname | janonymous: sorry. tox/venv was confusing me | 03:59 |
| notmyname | janonymous: bandit 0.17.3 works in tox | 03:59 |
| janonymous | notmyname: ohh great | 04:01 |
| *** kei_yama has quit IRC | 04:05 | |
| *** kei_yama_ has joined #openstack-swift | 04:05 | |
| *** mingdang1 has quit IRC | 04:07 | |
| *** silor1 has joined #openstack-swift | 04:09 | |
| *** dmorita has quit IRC | 04:13 | |
| *** silor1 has quit IRC | 04:14 | |
| *** silor has quit IRC | 04:14 | |
| *** silor has joined #openstack-swift | 04:16 | |
| *** psachin has joined #openstack-swift | 04:32 | |
| clayg | jrichli: maybe al and I were running with ssync or post_as_copy = false or something? | 04:33 |
| *** mingdang1 has joined #openstack-swift | 04:34 | |
| jrichli | clayg: I think its that I had an old conf file that didn't have realms defined correctly | 04:34 |
| clayg | jrichli: oic | 04:34 |
| jrichli | clayg: btw, both the fast-post and the post-as-copy tests had failed | 04:34 |
| clayg | i'd be nice if the container_sync probetests would detect if container synt was setup and bail accordingly :\ | 04:34 |
| clayg | so are probetests passing on master now or no? | 04:35 |
| clayg | you don't have to answer that - i'm bringing up my vsaio now | 04:36 |
| jrichli | well, i hadn't posted my results yet because I thought i had it set-up to run, but they get skipped. however, i ran a test individually and it succeeds. | 04:36 |
| jrichli | clayg: clearly, I need to learn how to fully setup container-sync so the tests don't get skipped now that I changed my bad conf file. But it will have to wait until tomorrow. I am interested to hear if yours pass. | 04:40 |
| jrichli | sgundur- make sure you don't have an outdated version of container-sync-realms.conf | 04:42 |
| *** sheel has joined #openstack-swift | 04:44 | |
| *** silor has quit IRC | 04:48 | |
| jrichli | one thought on all this: it could be that the patch I mentioned no longer supports using what some docs say is "the older allowed_sync_hosts way of syncing." but i haven't dug into what that means yet. | 04:55 |
| * jrichli is going sleep now | 04:55 | |
| *** silor has joined #openstack-swift | 04:57 | |
| *** nadeem has joined #openstack-swift | 05:05 | |
| *** kei_yama has joined #openstack-swift | 05:07 | |
| *** kei_yama_ has quit IRC | 05:07 | |
| *** dmorita has joined #openstack-swift | 05:14 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 05:14 |
| *** natarej has quit IRC | 05:16 | |
| *** natarej has joined #openstack-swift | 05:17 | |
| *** dmorita has quit IRC | 05:18 | |
| *** ukaynar has joined #openstack-swift | 05:22 | |
| *** mingdang1 has quit IRC | 05:26 | |
| *** mingdang1 has joined #openstack-swift | 05:26 | |
| *** mingdang1 has quit IRC | 05:32 | |
| *** ChubYann has quit IRC | 05:40 | |
| *** d0ugal has joined #openstack-swift | 05:52 | |
| *** _JZ_ has quit IRC | 05:58 | |
| *** klrmn has quit IRC | 06:15 | |
| *** rcernin has joined #openstack-swift | 06:26 | |
| *** ukaynar has quit IRC | 06:27 | |
| *** mmcardle has joined #openstack-swift | 06:27 | |
| *** jmccarthy has quit IRC | 06:29 | |
| *** jmccarthy has joined #openstack-swift | 06:29 | |
| *** dmorita has joined #openstack-swift | 06:36 | |
| *** dmorita has quit IRC | 06:42 | |
| *** SkyRocknRoll has joined #openstack-swift | 06:49 | |
| *** mariusv has quit IRC | 07:00 | |
| *** mariusv has joined #openstack-swift | 07:01 | |
| *** tesseract has joined #openstack-swift | 07:03 | |
| *** jamielennox is now known as jamielennox|away | 07:08 | |
| *** acoles_ is now known as acoles | 07:13 | |
| *** acoles is now known as acoles_ | 07:18 | |
| *** rledisez has joined #openstack-swift | 07:22 | |
| *** d0ugal has quit IRC | 07:23 | |
| *** d0ugal has joined #openstack-swift | 07:26 | |
| *** d0ugal has quit IRC | 07:26 | |
| *** d0ugal has joined #openstack-swift | 07:26 | |
| *** acoles_ is now known as acoles | 07:34 | |
| acoles | jrichli: clayg fwiw /test/probe/test_container_sync.py passes for me with post_as_copy true or false, rsync (not that I think that would make any difference) | 07:36 |
| acoles | I see some key errors in test_locked_container_dbs | 07:36 |
| *** bil has joined #openstack-swift | 07:47 | |
| bil | Hi can someone help me with joss auth issues? | 07:48 |
| *** jamielennox|away is now known as jamielennox | 07:49 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - cleanup encrypter error handling https://review.openstack.org/312662 | 07:50 |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - cleanup decrypter exception handling https://review.openstack.org/304806 | 07:50 |
| *** bil has quit IRC | 07:54 | |
| *** jordanP has joined #openstack-swift | 08:01 | |
| *** cbartz has joined #openstack-swift | 08:07 | |
| *** mvk has quit IRC | 08:16 | |
| *** daemontool__ is now known as daemontool | 08:30 | |
| *** nadeem has quit IRC | 08:36 | |
| *** jistr has joined #openstack-swift | 08:37 | |
| *** mvk has joined #openstack-swift | 08:45 | |
| *** mingdang1 has joined #openstack-swift | 09:05 | |
| *** eranrom has joined #openstack-swift | 09:14 | |
| eranrom | Yesterday, there has been a discussion on problems with container sync probe test. I may have found a problem, however, I am still confused as why this does not happen 'at the gate'. | 09:18 |
| eranrom | I suspect that every first call to get_info (proxy/controller/base.py) will update the cache but will return no info. | 09:19 |
| eranrom | thus on a second call the info is returned, but not on the first. | 09:19 |
| eranrom | this is basically why the probe tests fail. | 09:19 |
| eranrom | Now, the reason for the first call to always fail is the following: get_info creates an internal HEAD request for the container (in our case we are in the get_container_info code path), and does req.get_response(app) which in turn gets to GETorHEAD_base | 09:22 |
| eranrom | The code in get_info assumes that GETorHEAD_base would update the request environ to include the container info (which it does), however, | 09:23 |
| eranrom | the environment being updated by GETorHEAD_base does not seem to update the environmant of the request upon which we call req.get_resonse. | 09:24 |
| eranrom | My question is: when doing reg.get_response which calls the application do we expect the env of the calling request to get updated? | 09:24 |
| *** daemontool has quit IRC | 09:28 | |
| acoles | eranrom: hi! I don't have the answer to your question, but I didn't see the container sync probe tests fail (see scrollback) | 09:29 |
| acoles | eranrom: do you see them fail? | 09:29 |
| *** chlong has quit IRC | 09:30 | |
| *** silor has quit IRC | 09:35 | |
| eranrom | acoles: I do see this from time to time. | 09:37 |
| eranrom | acoles: again, it never happended on the "gate" | 09:37 |
| acoles | eranrom: how do you have container sync configured? the new way or "old way"? | 09:39 |
| mahatic | eranrom: notmyname earlier said this about gate: "probetest gates have never been "gates" (ie voting) because they are set up to run against the SAIO. and nobody has figured out how to do that with the gate images. there are probetests running in the community qa cluster though, but they aren't voting" | 09:40 |
| *** mittal has joined #openstack-swift | 09:52 | |
| *** acoles is now known as acoles_ | 09:52 | |
| *** acoles_ is now known as acoles | 09:54 | |
| mittal | I have a query: In our swift deployment, with no replication, we are observing errors 'Client disconnected without sending enough data' | 09:58 |
| mittal | not sure where to start tuning/debugging from ? | 09:59 |
| mittal | These are observed on PUT requests only | 09:59 |
| *** mvk has quit IRC | 10:03 | |
| *** mvk has joined #openstack-swift | 10:04 | |
| eranrom | acoles,: I am using SAIO, so I assume its the new way. | 10:05 |
| eranrom | acoles: I guess that using SAIO means I have container sync configured... | 10:07 |
| *** daemontool has joined #openstack-swift | 10:08 | |
| *** hosanai has quit IRC | 10:15 | |
| *** mittal has quit IRC | 11:10 | |
| *** silor has joined #openstack-swift | 11:16 | |
| *** ppai has joined #openstack-swift | 11:21 | |
| *** silor1 has joined #openstack-swift | 11:23 | |
| *** silor has quit IRC | 11:24 | |
| *** silor1 is now known as silor | 11:24 | |
| eranrom | mahatic: thanks! I am still puzzled by the inconsistency failure of those tests. Lets see what the west side of the world has to say :-) | 11:24 |
| *** mingdang1 has quit IRC | 11:25 | |
| *** NM has joined #openstack-swift | 11:40 | |
| *** mingdang1 has joined #openstack-swift | 11:56 | |
| *** km has quit IRC | 12:01 | |
| *** ozialien10 has quit IRC | 12:08 | |
| *** ozialien10 has joined #openstack-swift | 12:09 | |
| acoles | eranrom: i have a memory of needing to set up container-sync-realms.conf as an extra step to SAIO. But it could be the SAIO docs have been updated since then to include it. | 12:20 |
| *** Lickitysplitted has quit IRC | 12:22 | |
| *** Lickitysplitted has joined #openstack-swift | 12:22 | |
| acoles | eranrom: Request.get_response passes the request environ to the app, and returns a response whose environ is that of the Request instance i.e. same environ | 12:24 |
| *** kei_yama has quit IRC | 12:34 | |
| eranrom | acoles,: setting up container-sync-realms.conf is now part of the SAIO setup. What you are saying about Request.get_response is also my understanding. The issue, however, is that the request env does not get changed by the call (which is the assumption made by get_info). | 12:36 |
| eranrom | more specifically, printing the request env before and after call_application shows the same env. Interestingly printing the env inside GETorHEAD_base does show the required change. Will try to dig dipper. | 12:38 |
| *** pauloewerton has joined #openstack-swift | 12:50 | |
| *** [1]eranrom has joined #openstack-swift | 12:55 | |
| *** eranrom has quit IRC | 12:55 | |
| *** [1]eranrom is now known as eranrom | 12:55 | |
| *** asettle has joined #openstack-swift | 13:03 | |
| *** dmorita has joined #openstack-swift | 13:04 | |
| *** dmorita has quit IRC | 13:09 | |
| *** asettle has quit IRC | 13:15 | |
| *** daemontool_ has joined #openstack-swift | 13:21 | |
| *** links has quit IRC | 13:23 | |
| *** daemontool has quit IRC | 13:24 | |
| *** tongli has joined #openstack-swift | 13:30 | |
| *** esker has quit IRC | 13:34 | |
| *** klamath has joined #openstack-swift | 13:37 | |
| pdardeau | good morning | 13:41 |
| *** cdelatte has joined #openstack-swift | 13:44 | |
| openstackgerrit | Andreas Jaeger proposed openstack/swift: List system dependencies for running common tests https://review.openstack.org/298313 | 13:50 |
| *** mingdang1 has quit IRC | 13:51 | |
| *** sgundur- has joined #openstack-swift | 13:51 | |
| *** sgundur- has left #openstack-swift | 13:55 | |
| *** sgundur- has joined #openstack-swift | 13:55 | |
| *** ametts has joined #openstack-swift | 13:58 | |
| *** _JZ_ has joined #openstack-swift | 14:01 | |
| *** mwheckmann has joined #openstack-swift | 14:03 | |
| mwheckmann | hello. Quick and dumb question: Any risk in *lowering* the num of replicas for a ring? | 14:05 |
| *** esker has joined #openstack-swift | 14:05 | |
| pdardeau | mwheckmann: it's a tradeoff of data durability and cost of storage. no right answer for all cases | 14:07 |
| pdardeau | mwheckmann: or were you asking about modifying an existing ring? | 14:08 |
| mwheckmann | pdardeau: exactly :) | 14:08 |
| mwheckmann | I wanted to know from a safety aspect. | 14:09 |
| pdardeau | mwheckmann: for modifying an existing ring? | 14:09 |
| mwheckmann | Wondering if there is any limitation, that makes reducing the number of replicas (vs increasing them) more risky. | 14:09 |
| mwheckmann | yes, it's an existing ring in PROD | 14:10 |
| mwheckmann | currently have 9 replicas across 3 regions. | 14:10 |
| mwheckmann | EMEA east, EMEA west and NCSA | 14:11 |
| mwheckmann | I was hoping to lower my number of replicas down to 6 | 14:11 |
| mwheckmann | from 9 | 14:11 |
| pdardeau | mwheckmann: that's a really tricky thing. i'll defer that to others here such as acoles, cschwede, clayg | 14:11 |
| mwheckmann | pdardeau: I can't find any documentation that says otherwise, but I do get warnings on the rebalance operation. | 14:12 |
| acoles | pdardeau: good morning. I'll defer to others with more experience in that area. | 14:12 |
| mwheckmann | Let's say I'm a little more hesitant to put that ring into PROD. | 14:12 |
| *** esker has quit IRC | 14:20 | |
| pdardeau | mwheckmann: how were you planning to get the reduction of replicas? | 14:20 |
| *** esker has joined #openstack-swift | 14:21 | |
| mwheckmann | pdardeau: what do you mean? I was just planning on running "swift-ring-builder <ring>.builder set_replicas 6" | 14:26 |
| mwheckmann | in fact, I did run it already but haven't put it into PROD | 14:26 |
| mwheckmann | pdardeau: if you're wondering why I want to do this, it's because PUT quorum for 9 replicas is 5. I'd like to get that down to 3. | 14:29 |
| ahale | you could try loading both rings up and see what changes per part - like how many replicas per region before and after | 14:29 |
| mwheckmann | ahale: yes, I was thinking of looking into that with swift-get-nodes | 14:30 |
| *** esker has quit IRC | 14:33 | |
| *** esker has joined #openstack-swift | 14:34 | |
| mwheckmann | ahale: thanks | 14:34 |
| mwheckmann | here are some of the warnings I received on rebalance: swift/common/ring/builder.py:1369: RingValidationWarning: All devices in tier (1, 1, '10.131.0.12') already contain a replica "contain a replica" % (tier,))) | 14:41 |
| mwheckmann | the one above just happened once. The rest all reseble the following: swift/common/ring/builder.py:618: RingValidationWarning: The partition 85418 has been assigned to duplicate devices [6, 9, 1, 1, 5, 14] part, devs_for_part))) | 14:42 |
| *** jamie_h has joined #openstack-swift | 14:42 | |
| mwheckmann | of course, this is just an intermidiate state. I'll have to run rebalance again. | 14:42 |
| mwheckmann | so my guess is that things will fall into place. | 14:43 |
| cschwede | mwheckmann: note that you have to use upstream master to be able to use 6 replicas to get the quorum down to 3 | 14:47 |
| cschwede | mwheckmann: that change just landed a few days ago: https://github.com/openstack/swift/commit/29544a9e | 14:47 |
| cschwede | mwheckmann: so if you run one of the stable releases, you have to use 5 replicas to get down to a quorum of 4 | 14:48 |
| cschwede | s/4/3/ | 14:48 |
| cschwede | typo... | 14:48 |
| cschwede | mwheckmann: about the warnings - the ring-builder will only (re-)move one copy at a time, so if you change replicas in steps of 1 you probably won’t get warnings (if the ring was balanced well before) | 14:49 |
| mwheckmann | cschwede: understood. thanks for the info. Do you know anything about any bugs/risks when lowering the number of replicas on a ring in PROD? | 14:49 |
| cschwede | mwheckmann: well, besides the risk of a lower durability - i’m not aware of any bugs atm; the replicators should fix this. personally i wouldn’t be worried to have „only“ 5 or 6 replicas | 14:52 |
| cschwede | mwheckmann: but to be fully sure everything works as expected i would simply create another storage policy, create some data in it and then decrease the replicas of that policy | 14:52 |
| *** pcaruana has joined #openstack-swift | 14:53 | |
| mwheckmann | cschwede: good idea. | 14:54 |
| *** esker has quit IRC | 14:54 | |
| notmyname | good morning | 14:54 |
| *** esker has joined #openstack-swift | 14:55 | |
| mwheckmann | cschwede: I'm not worried about the durability. More of any bugs or limitations in dropping the replica count on a ring in PROD since it's probably an operation that isn't done as frequently | 14:55 |
| sgundur- | notmyname: morning, yesterday when I rechecked with the current master + 311899 ; I ended up getting the same errors around the container_sync | 15:04 |
| *** Jeffrey4l_ has quit IRC | 15:05 | |
| sgundur- | notmyname: also my saio setup is on rpc VM | 15:06 |
| *** psachin has quit IRC | 15:07 | |
| *** ukaynar has joined #openstack-swift | 15:08 | |
| mwheckmann | notmyname: sorry to ask you point blank, but is there any known risk in reducing replica count on a ring? (I'm talking about a production ring of course) | 15:09 |
| notmyname | if you reduce a replica count, you'll basically immediately get the new ring's availability characteristics (reduced number of primary nodes) for reads, you'll immediately get the (likely) faster writes from a reduced quorum, but it will take a while for the replication processes to sweep up the old replicas (ie reclaim space) | 15:12 |
| notmyname | I wouldn't guess that there would be any adverse affect from reducing replicas | 15:12 |
| notmyname | (but that's rather untested. just a guess. I don't /think/ I'm forgetting something, but I wouldn't know if I am ;-) | 15:13 |
| notmyname | probably the way to test it is using the dispersion report | 15:13 |
| dfg_ | i thought you could use the fractional replicas thing to do this incrementally- or did somebody already say that | 15:14 |
| mwheckmann | notmyname: thanks. | 15:14 |
| notmyname | not sure if someone said that, but yeah, use fractional replicas to gradually adjust it down | 15:15 |
| notmyname | 4 -> 3.9 -> 3.8 -> ... -> 3.1 -> 3 | 15:15 |
| mwheckmann | ok. I will try that. | 15:16 |
| notmyname | when increasing replicas, you gotta take into account the rebalancing that happens. the system thinks data should be somewhere but it hasn't made it over there yet | 15:16 |
| notmyname | but with reducing replicas, you're basically jsut immediately forcing over-replication and it should be find from an anvailability point of view | 15:17 |
| mwheckmann | got it | 15:17 |
| notmyname | so while you pretty much must use fractional replicas to increase the replica count, I'm not sure that's strictly necessary (or gains you much at all) when reducing replicas | 15:18 |
| notmyname | but it certainly won't hurt anything to reduce replicas in small steps | 15:18 |
| mwheckmann | Wondering about this commit: https://github.com/openstack/swift/commit/29544a9e While it clearly helps in the 2 region case, I don't see it helping if we have more than two. Does write affinity take precedence over the quorum? | 15:20 |
| mwheckmann | i.e will it write to the handoff partitions? | 15:20 |
| mwheckmann | Any reason that the proxy quorum couldn't be configurable? | 15:20 |
| mwheckmann | (as much as I dislike the idea of adding more knobs) | 15:21 |
| notmyname | write affinity and quorum are orthogonal | 15:22 |
| notmyname | quorum is how many must be written. affinity is where they will be written | 15:22 |
| *** chuck__ is now known as zul | 15:24 | |
| *** arch-nemesis has joined #openstack-swift | 15:24 | |
| *** AJaeger has joined #openstack-swift | 15:24 | |
| AJaeger | notmyname: are you around to discuss other-requirements? | 15:26 |
| *** wshao has joined #openstack-swift | 15:26 | |
| notmyname | AJaeger: sure | 15:27 |
| AJaeger | since we both worked on this, should we merge efforts? I prefer my change since it includes the tox.ini setup and the now added "test" selector - but I'm fine if you take it over and add a co-authored-by... | 15:28 |
| AJaeger | I can also abandon mine. What do you prefer? | 15:28 |
| notmyname | AJaeger: I heard about other-reqs and thought it sounded interesting and cobbled together something, but I have no particular dog in the fight. seems like yours has more stuff (tox.ini and doc updates) and, really, I was just guessing at what to put in mine | 15:29 |
| notmyname | so, I'm fine to go with yours | 15:30 |
| AJaeger | notmyname: I was guessing as well ;) But tried to minimize... | 15:30 |
| notmyname | my only question is about the differences between the two files | 15:30 |
| AJaeger | Mine seems to work - otherwise it wouldn't pass the tests. | 15:30 |
| notmyname | AJaeger: which patch is yours? | 15:30 |
| notmyname | yeah, mine too :-) | 15:30 |
| AJaeger | notmyname: https://review.openstack.org/298313 | 15:30 |
| AJaeger | notmyname: you're adding a few packages that are in the base image, so don't need to be installed. | 15:31 |
| AJaeger | Like python | 15:31 |
| notmyname | what does the "test" thing do? | 15:31 |
| notmyname | might be needed on 16.04 :-) | 15:31 |
| AJaeger | notmyname: see http://lists.openstack.org/pipermail/openstack-infra/2016-May/004232.html and followups | 15:32 |
| notmyname | and yeah, the line of what's a dependency or not is fuzzy for me. where do we stop? | 15:32 |
| AJaeger | And also https://review.openstack.org/314574 | 15:32 |
| AJaeger | notmyname: it doesn't matter if you add more ;) | 15:32 |
| AJaeger | if they're installed already, it's good | 15:33 |
| AJaeger | notmyname: the test comes from https://review.openstack.org/314571 - we plan to run "bindep test", so that you can specify both runtime dependencies and test dependencies. See also the comments by lifeless on 298313 | 15:33 |
| notmyname | seems weird to have "test" on everything though | 15:34 |
| AJaeger | yeah ;) | 15:35 |
| *** pgbridge has joined #openstack-swift | 15:35 | |
| AJaeger | but if you add back python, it would be a base requirement | 15:35 |
| notmyname | but python-dev is only for testing? | 15:35 |
| *** esker has quit IRC | 15:35 | |
| notmyname | I'm not sure that's true (but I don't have data either way) | 15:35 |
| notmyname | well, no data aside from it just doesn't really seem to work without the -dev package | 15:36 |
| notmyname | ie for building other dependencies | 15:36 |
| notmyname | so that seems more than a "test" thing to me | 15:36 |
| *** esker has joined #openstack-swift | 15:36 | |
| notmyname | but now we're into the /me is guessing territory again | 15:36 |
| AJaeger | exactly - let's see what lifeless says when he reviews, you might want to talk with him about it - and with fungi. best on #openstack-infra | 15:37 |
| AJaeger | and feel free to take over my change - or merge it and update the file afterwards ;) | 15:37 |
| notmyname | no no. other way around :-) | 15:37 |
| clarkb | python-dev is used to link against python | 15:37 |
| clarkb | eg when liberasure coding python lib is built | 15:37 |
| notmyname | AJaeger: you should update yours to include anything in mine that's good, and I should abandon mine :-) | 15:38 |
| AJaeger | clarkb: so, should it be in test or in the base? | 15:38 |
| clarkb | I would put it in base | 15:38 |
| clarkb | since swift features rely on iy | 15:38 |
| clarkb | not just tests | 15:39 |
| AJaeger | notmyname: ok, will do. | 15:39 |
| AJaeger | clarkb: Ok, will change. | 15:39 |
| notmyname | AJaeger: oh, and in yours, jerasure isn't needed at all | 15:39 |
| * AJaeger will send a new version later | 15:39 | |
| AJaeger | notmyname: great! | 15:40 |
| notmyname | so should any of them be test? | 15:40 |
| notmyname | all of them are for actual basic swift features and not just for testing | 15:40 |
| AJaeger | notmyname: let me send of an email on openstack-infra in reply to lifeless's latest one - are you on the list or should I CC you? | 15:41 |
| notmyname | AFAIK we have zero non-python dependencies that are used only for testing | 15:41 |
| notmyname | AJaeger: I don't think I'm on that list | 15:41 |
| AJaeger | ok, I'll CC you... | 15:41 |
| notmyname | thanks | 15:41 |
| mwheckmann | notmyname: so if write affinity and quorum are orthogonal: With 6 replicas across 3 regions (2 replicas per region) quorum would be 4. Let's say I'm writing on region 1 and my affinity there is set to region 1 as well. It will write 4 replicas in region 1 (on the handoff devices?) even if normally my ring specified 2 replicas in region 1? | 15:45 |
| notmyname | you'll write 6 in region 1 and require that 4 be successfully written. | 15:46 |
| AJaeger | notmyname: mail send, I'll be back online later and update the change... | 15:46 |
| mwheckmann | ok. So then write affinity takes precedence over what the normal behaviour of the ring would be | 15:47 |
| notmyname | also, 6 replicas + 3 regions = 2x per region if and only if the regions are exactly the same size (ie exact same total weight) and your cluster is steady-state healthy. ie only if you're lucky | 15:47 |
| mwheckmann | notmyname: of course | 15:48 |
| notmyname | AJaeger: thanks | 15:48 |
| mwheckmann | I do always make sure that my rings are well balanced :) | 15:48 |
| *** tesseract has quit IRC | 15:52 | |
| * notmyname needs to get ready and go to the office | 15:54 | |
| *** wshao has quit IRC | 16:03 | |
| openstackgerrit | Andreas Jaeger proposed openstack/swift: List system dependencies for running common tests https://review.openstack.org/298313 | 16:10 |
| *** daemontool_ has quit IRC | 16:13 | |
| openstackgerrit | Paul Dardeau proposed openstack/swift: resurrect gholt blog posts on building consistent hashing ring https://review.openstack.org/314270 | 16:16 |
| openstackgerrit | Paul Dardeau proposed openstack/swift: Prose lines wrapped at 70 characters https://review.openstack.org/314660 | 16:17 |
| *** asettle has joined #openstack-swift | 16:18 | |
| *** asettle has quit IRC | 16:23 | |
| *** lyrrad has joined #openstack-swift | 16:24 | |
| *** esker has quit IRC | 16:25 | |
| *** ppai has quit IRC | 16:25 | |
| *** esker has joined #openstack-swift | 16:26 | |
| *** tqtran has joined #openstack-swift | 16:26 | |
| *** chuck__ has joined #openstack-swift | 16:30 | |
| *** joeljwright has quit IRC | 16:31 | |
| *** zul has quit IRC | 16:31 | |
| *** chuck__ is now known as zul | 16:31 | |
| *** cbartz has quit IRC | 16:36 | |
| *** diogogmt has joined #openstack-swift | 16:43 | |
| *** CaioBrentano has joined #openstack-swift | 16:46 | |
| *** flaper87 has quit IRC | 16:46 | |
| *** flaper87 has joined #openstack-swift | 16:46 | |
| *** cdelatte has quit IRC | 16:46 | |
| *** cdelatte has joined #openstack-swift | 16:47 | |
| *** ukaynar has quit IRC | 16:47 | |
| notmyname | pdardeau: nice. thanks (for the line-wrapping) | 16:50 |
| notmyname | pdardeau: also, now if you're measured by "lines of docs proposed" your score went up by 5x or something ;-) | 16:50 |
| *** nadeem has joined #openstack-swift | 16:50 | |
| notmyname | pdardeau: I've got a lot of meetings today, but I'll try to give it a review later today | 16:51 |
| *** nadeem has quit IRC | 16:53 | |
| *** nadeem has joined #openstack-swift | 16:54 | |
| *** delattec has joined #openstack-swift | 16:56 | |
| *** mvk has quit IRC | 16:57 | |
| pdardeau | notmyname: lol, thx | 16:58 |
| notmyname | bkeller`: ping | 16:58 |
| bkeller` | hi | 16:59 |
| pdardeau | notmyname: i think your comment about openstack clippy has been burned into my long-term memory banks (it's a keeper) | 16:59 |
| notmyname | pdardeau: lol | 16:59 |
| *** klrmn has joined #openstack-swift | 16:59 | |
| notmyname | bkeller`: so I just saw the bug you assinged yourself in LP (bug 1576889) | 16:59 |
| openstack | bug 1576889 in OpenStack Object Storage (swift) " change default ports for servers" [Undecided,Confirmed] https://launchpad.net/bugs/1576889 - Assigned to Bryan Keller (kellerbr) | 16:59 |
| *** cdelatte has quit IRC | 16:59 | |
| notmyname | bkeller`: that's the doc bug that was auto-created? | 16:59 |
| bkeller` | yeah, was that already fixed? | 17:00 |
| notmyname | bkeller`: AFAICT, that's actually already done in sgundur-'s original patch | 17:00 |
| *** esker has quit IRC | 17:00 | |
| notmyname | I'm guessing she added the DocImpact tag because, well, it impacted the docs ( sgundur- you did not do anything wrong there) | 17:00 |
| *** daemontool_ has joined #openstack-swift | 17:00 | |
| bkeller` | ah, ok | 17:00 |
| notmyname | bkeller`: but yeah, if you can give the code a cursory overview to see the updated port ranges etc, that would be great. and, if it's done, and if you can, mark it as resolved/nonissue/closed? | 17:01 |
| bkeller` | I think I mistook some of the other services that were still running on 60xx then | 17:01 |
| bkeller` | sure | 17:01 |
| *** jordanP has quit IRC | 17:03 | |
| *** rledisez has quit IRC | 17:04 | |
| *** jmccarthy1 has joined #openstack-swift | 17:14 | |
| *** BAKfr has quit IRC | 17:20 | |
| *** BAKfr has joined #openstack-swift | 17:20 | |
| *** jistr has quit IRC | 17:21 | |
| notmyname | bkeller`: ah cool. it's all taken care of then? | 17:22 |
| bkeller` | yeah, there was nothing for me to do | 17:23 |
| sgundur- | notmyname: ha :) , bkeller` thanks | 17:23 |
| *** ChubYann has joined #openstack-swift | 17:24 | |
| sgundur- | notmyname, bkeller` I had added that tag , but didn't know it abt the auto-created that bug | 17:25 |
| *** daemontool_ has quit IRC | 17:25 | |
| notmyname | yeah, DocImpact used to be the way to tell the docs team "hey something changed". now it will auto-create a bug in our own project | 17:26 |
| notmyname | since we're pretty good about actually enforcing "docs land with code", the docimpact is not nearly as useful | 17:26 |
| *** lcurtis has joined #openstack-swift | 17:46 | |
| *** asettle has joined #openstack-swift | 17:49 | |
| *** asettle has quit IRC | 17:50 | |
| *** acoles is now known as acoles_ | 17:54 | |
| *** garthb has joined #openstack-swift | 17:57 | |
| *** chuck__ has joined #openstack-swift | 18:06 | |
| *** zul has quit IRC | 18:07 | |
| *** chuck__ is now known as zul | 18:07 | |
| *** ukaynar has joined #openstack-swift | 18:11 | |
| *** rcernin has quit IRC | 18:13 | |
| *** mvk has joined #openstack-swift | 18:17 | |
| *** thumpba has joined #openstack-swift | 18:25 | |
| *** gyee has joined #openstack-swift | 18:34 | |
| *** janonymous_ has joined #openstack-swift | 18:35 | |
| lifeless | AJaeger: notmyname: you wouldn't have 'test' on everything, only on those things which are not runtime deps. | 18:40 |
| AJaeger | lifeless: then my change https://review.openstack.org/#/c/298313 should be fine. thanks | 18:42 |
| *** janonymous_ has quit IRC | 18:42 | |
| *** janonymous_ has joined #openstack-swift | 18:42 | |
| *** rcernin has joined #openstack-swift | 18:43 | |
| lifeless | AJaeger: wel no, because the dev packages are not runtime deps | 18:44 |
| lifeless | AJaeger: imagine you're building wheels | 18:44 |
| *** thumpba has quit IRC | 18:44 | |
| *** thumpba has joined #openstack-swift | 18:46 | |
| AJaeger | lifeless: clarkb, notmyname discussed a bit earlier... | 18:46 |
| AJaeger | lifeless: but if I'm not building wheels, then those are required. | 18:46 |
| clarkb | they are install deps | 18:46 |
| clarkb | not test deps | 18:47 |
| AJaeger | so, how to classify these then? | 18:47 |
| lifeless | ffi-devel is a build dep | 18:49 |
| lifeless | libffi6 is a runtime dep | 18:49 |
| AJaeger | agreed, lifeless | 18:50 |
| AJaeger | So, how do we add these to profiles? | 18:50 |
| AJaeger | We have the base profile that includes runtime deps - and we discussed the test profile for gating. So, build deps go to the test profile? | 18:51 |
| lifeless | AJaeger: I'm saying I think that the runtime deps should be in the default profile, and build deps should not | 18:51 |
| lifeless | AJaeger: since testing (for us) is intrinsically a source operation, putting the nonruntime deps needed for testing in a test profile is what I have suggested | 18:52 |
| clarkb | but then anyone installing from source which is the target audience will fail | 18:52 |
| lifeless | clarkb: how will they fail? | 18:52 |
| clarkb | install wont work | 18:52 |
| lifeless | clarkb: bindep isn't invoked by pip or anything else; they're going to be following docs we write | 18:52 |
| clarkb | if everyone using bindep must install test deps anyways why distinguish at all | 18:54 |
| AJaeger | clarkb, lifeless: I just updated the infra-manual section on bindep, see https://review.openstack.org/314574 | 18:54 |
| AJaeger | That reflects what lifeless just said IMHO - and we could make clearer that developer or people building from source need this test environment. | 18:55 |
| AJaeger | What do you think? | 18:55 |
| clarkb | I dislike it, I think that regardless of why someone is installing eg swift that the default bindep list should just work | 18:56 |
| clarkb | maybe they are running tests, maybe this is how they deploy, I don't care. Pip install after bindep runs should just work | 18:57 |
| AJaeger | We should discuss together with fungi - can we move to #openstack-infra and continue after the meeting | 18:57 |
| tdasilva | here's patchset 1 to merge pyeclib/liberasurecode to openstack gerrit: https://review.openstack.org/#/c/314724/ | 18:58 |
| patchbot | tdasilva: patch 314724 - openstack-infra/project-config - Migrating liberasurecode and pyeclib projects | 18:58 |
| *** esker has joined #openstack-swift | 18:59 | |
| notmyname | tdasilva: thanks! | 19:00 |
| notmyname | FYI, the infra team meeting is starting now in -meeting, and one of the topics is about what we'll need to work together on for testing/CI/gate/etc for golang stuff | 19:01 |
| notmyname | mostly I'm expecting to get a list of stuff to ask instead of get actual answers. if you're interested, it's starting now | 19:02 |
| openstackgerrit | Paul Dardeau proposed openstack/swift: resurrect gholt blog posts on building consistent hashing ring https://review.openstack.org/314270 | 19:07 |
| *** pcaruana has quit IRC | 19:12 | |
| *** acoles_ is now known as acoles | 19:13 | |
| *** jmccarthy1 has quit IRC | 19:19 | |
| *** SkyRocknRoll has quit IRC | 19:28 | |
| notmyname | here's some initial notes from the -infra meeting https://etherpad.openstack.org/p/golang-infra-issues-to-solve | 19:36 |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - don't break conditional GETs on unencrypted objects https://review.openstack.org/314740 | 19:37 |
| acoles | phew that was fun ^^ ! | 19:38 |
| acoles | notmyname: I updated crypto priority reviews to add that ^^ | 19:40 |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - don't break conditional GETs on unencrypted objects https://review.openstack.org/314740 | 19:43 |
| *** mmcardle has quit IRC | 19:44 | |
| lifeless | clarkb: honestly, I don't have the energy to argue, do whatever makes sense | 19:44 |
| *** acoles is now known as acoles_ | 19:45 | |
| lifeless | clarkb: I was asked for my opinion, I've given it | 19:45 |
| *** rcernin has quit IRC | 19:51 | |
| *** daemontool_ has joined #openstack-swift | 19:51 | |
| *** tongli has quit IRC | 19:53 | |
| *** bsdkurt has quit IRC | 19:58 | |
| *** rcernin has joined #openstack-swift | 19:58 | |
| *** janonymous_ has quit IRC | 19:59 | |
| clarkb | lifeless: as did I :) out of curiosity do you not expect most bindep users to be installing from source? | 20:03 |
| notmyname | acoles_: thanks | 20:04 |
| *** daemontool_ has quit IRC | 20:06 | |
| lifeless | clarkb: I know a bunch of deployers that start from source but deploy binaries | 20:07 |
| lifeless | clarkb: (both wheels and debs) | 20:08 |
| lifeless | clarkb: like upper-constraints, I expect other-requirements to provide input to their deployment metadata, whether the actual things copied onto their servers are source/wheels/debs/rpms | 20:08 |
| clarkb | seems like we should be able to achieve those goals while also making default just work | 20:10 |
| clarkb | maybe annotations that don't impact the default run? | 20:11 |
| torgomatic | notmyname: I tried to weigh in on the whole golang thing. Hopefully I managed to convey that read() takes a long time and blocks the calling thread. http://lists.openstack.org/pipermail/openstack-dev/2016-May/094549.html | 20:12 |
| notmyname | torgomatic: I saw. thanks :-) | 20:12 |
| lifeless | possibly, it would require a bit of a back-to-basic design review of the ux/minilanguage. | 20:12 |
| lifeless | torgomatic: it sounds like go does exactly like what you did in Python | 20:14 |
| lifeless | torgomatic: a threadpool for syscalls | 20:14 |
| *** bsdkurt has joined #openstack-swift | 20:14 | |
| lifeless | torgomatic: (As a data point, in case its interesting, squid does exactly the same thing with one of its drivers [it has many :P]) - its C++ these days, written in a nonblocking event loop style. | 20:15 |
| torgomatic | lifeless: pretty much; it just does it transparently and quickly, whereas the stuff I did was neither one ;) | 20:15 |
| tdasilva | wow, it's hard to follow the convo in #openstack-meeting | 20:42 |
| *** suyash has joined #openstack-swift | 20:46 | |
| *** esker has quit IRC | 20:47 | |
| *** silor has quit IRC | 20:49 | |
| notmyname | tdasilva: the TC meeting? yeah, I didn't realize golang would be a topic there. I happened to see my client highlight, and i'm glad I did | 20:52 |
| tdasilva | notmyname: yes, interesting meeting. | 20:53 |
| *** thumpba has quit IRC | 21:02 | |
| *** garthb has quit IRC | 21:09 | |
| *** mingdang1 has joined #openstack-swift | 21:11 | |
| jrichli | torgomatic: nice reply. good to see some more analytical facts on the thread - less conjecture and pre-determined judgements | 21:12 |
| *** pauloewerton has quit IRC | 21:15 | |
| *** ukaynar has quit IRC | 21:15 | |
| *** ametts has quit IRC | 21:17 | |
| *** garthb has joined #openstack-swift | 21:17 | |
| *** NM has quit IRC | 21:31 | |
| pdardeau | that's a lot to digest in TC meeting minutes | 21:34 |
| *** daemontool_ has joined #openstack-swift | 21:35 | |
| pdardeau | i'd like to ask -- how did hummingbird/go flip from not being a thing to being a big topic of interest overnight? | 21:35 |
| notmyname | well sit back and grandpappy will tell you a story of how things used to be..... | 21:36 |
| pdardeau | btw, i'm not opposed to go (in fact, it has some appeal if it doesn't include pip!) | 21:36 |
| notmyname | clarkb: did you see sam's recent message to the thread about some of the technical reasons of golang over python? I only wanted to point that out because we aren't blaming eventlet or trying to replace eventlet with golang | 21:38 |
| clarkb | yes, that is part of where I got that from (but also reading the designate code it seems like a huge part of the potential slowness there is just poor eventlet and related code) | 21:39 |
| clarkb | every dns query uses a brand new greenthread which in turn uses a brand new db session | 21:39 |
| zaitcev | pdardeau: Basically redbo is a genius. He went away for a weekend and basically came back hummingbird's object server fully functional. His biggest thing was writing a pickle parser in Go (that's where my own Swift-in-C faltered). | 21:40 |
| zaitcev | pdardeau: from there it was a matter of gradually adding functionality step by step, such as replication. | 21:40 |
| clarkb | notmyname: I interpretted that email as eventlet makes these things hard (and not necessarily python) | 21:41 |
| notmyname | pdardeau: the summary is that we presented our plan to use go, there's some necessary follow up from some openstack cross-project stuff (like docs, infra, etc), and the resulting "conversation" has a lot of people trying to grok the whole situation and its consequences. and so there's a lot of people who want to say something about that | 21:43 |
| pdardeau | zaitcev: thanks for the background | 21:43 |
| pdardeau | zaitcev: the Swift-in-C sounds interesting. is it in github? (not trying to stir the pot, more just from curiosity) | 21:44 |
| notmyname | so yeah, redbo (and then dfg_ and nadeem and others) have been working on it for a while, but the http://lists.openstack.org/pipermail/openstack-dev/2016-May/093795.html message (and the one linked there) are what started everything in the last few days | 21:44 |
| zaitcev | pdardeau: https://github.com/zaitcev/swift-oserver - but there's nothing useable there, except that it has its own webserver that I stole from jgarzik's tabled. | 21:45 |
| pdardeau | notmyname: right. i'm more curious in what prompted it to be a big discussion at summit | 21:45 |
| notmyname | pdardeau: ah. I guess you can blame that on me :-) | 21:45 |
| zaitcev | or, actually not even that. it was the plan | 21:46 |
| clarkb | notmyname: specifically lots of discussion about IO, and read(), and slow disks, which are not python problems they exist for Go too, then the explanation of threadpool overhead (whih is an eventlet issue and not necessarily a python issue) | 21:46 |
| notmyname | it's something that as a community we've been talking about for a year or 18 months now. so based on the rax results + the current state of hummingbird + what other clusters are seeing as issues, it was time to make a go/no-go decision about the golang work | 21:46 |
| pdardeau | notmyname: heh, go/no-go | 21:47 |
| clarkb | certainly one way of addressing that is to use a different language end or VM/runtime | 21:50 |
| *** cdelatte has joined #openstack-swift | 21:52 | |
| *** delattec has quit IRC | 21:54 | |
| *** cdelatte has quit IRC | 21:55 | |
| *** daemontool_ has quit IRC | 21:55 | |
| *** cdelatte has joined #openstack-swift | 21:59 | |
| pdardeau | zaitcev: i had recently looked into doing something similar for c++, d, java, nim, and scala https://github.com/pauldardeau/swift-babel | 22:01 |
| zaitcev | pdardeau: trere's a pickle for C++ BTW | 22:04 |
| *** cdelatte has quit IRC | 22:08 | |
| *** nadeem has quit IRC | 22:09 | |
| *** mwheckmann has quit IRC | 22:10 | |
| *** ukaynar has joined #openstack-swift | 22:11 | |
| *** cdelatte has joined #openstack-swift | 22:11 | |
| *** _JZ_ has quit IRC | 22:13 | |
| notmyname | pdardeau: yeah! why aren't we rewriting swift in nim!? | 22:14 |
| *** sheel has quit IRC | 22:15 | |
| pdardeau | notmyname: have you looked at nim? it's a cool language, especially if you like the intersection of python and pascal | 22:16 |
| notmyname | pdardeau: yeah, a little bit, but not enough to get much working. I was working on a swift directory uploader | 22:16 |
| notmyname | pdardeau: (doesn't actually work) https://gist.github.com/notmyname/bd5e01a42831b46b80109686213c7842 | 22:17 |
| notmyname | pdardeau: some of the operator overloading in nim is kinda mind-bending | 22:18 |
| notmyname | pdardeau: FWIW, that might compile with either num 0.10.2 or 0.11.2 | 22:18 |
| notmyname | *nim | 22:18 |
| notmyname | but I think they've made breaking updates to the language since then | 22:19 |
| *** CaioBrentano has quit IRC | 22:19 | |
| pdardeau | notmyname: swift_pusher? spoken like a true PTL ;-) | 22:21 |
| notmyname | ...and that's why I hadn't published it before (and why I just deleted it) | 22:21 |
| notmyname | had my RAX password in it. oops. all updated now | 22:21 |
| notmyname | passwords reset | 22:21 |
| *** mingdang_ has joined #openstack-swift | 22:23 | |
| *** mingdang1 has quit IRC | 22:24 | |
| *** delattec has joined #openstack-swift | 22:24 | |
| notmyname | ok, sanitized version https://gist.github.com/notmyname/af8c99f04e7b8bedcff3c44827d602d7 | 22:24 |
| *** pgbridge has quit IRC | 22:25 | |
| pdardeau | notmyname: in your defense, it did say 'discard' :) | 22:26 |
| notmyname | pdardeau: the way I do that in python scripts is like https://github.com/notmyname/cf_dropbox/blob/master/cf_drop.py#L10 | 22:26 |
| notmyname | then i have a global cf_auth.py in my site-packages that has my creds | 22:27 |
| *** cdelatte has quit IRC | 22:27 | |
| notmyname | pdardeau: and the nice things about using a password manager is (1) you wouldn't guess that password and (2) I'm not using it for *anything* else | 22:28 |
| *** pgbridge has joined #openstack-swift | 22:38 | |
| *** ukaynar has quit IRC | 22:41 | |
| openstackgerrit | John Dickinson proposed openstack/swift: resurrect gholt blog posts on building consistent hashing ring https://review.openstack.org/314270 | 22:41 |
| notmyname | pdardeau: I simply added python syntax highlighting to the code blocks | 22:41 |
| mattoliverau | morning | 22:47 |
| pdardeau | notmyname: nice! thx | 22:48 |
| pdardeau | good morning mattoliverau | 22:48 |
| mattoliverau | pdardeau: hey man | 22:49 |
| *** mingdang_ has quit IRC | 22:51 | |
| *** lcurtis has quit IRC | 22:56 | |
| *** jamielennox is now known as jamielennox|away | 23:13 | |
| *** delatte has joined #openstack-swift | 23:18 | |
| *** d0ugal has quit IRC | 23:21 | |
| *** delattec has quit IRC | 23:21 | |
| *** jamielennox|away is now known as jamielennox | 23:21 | |
| *** diogogmt has quit IRC | 23:23 | |
| *** km has joined #openstack-swift | 23:24 | |
| *** rcernin has quit IRC | 23:28 | |
| *** arch-nemesis has quit IRC | 23:30 | |
| *** kei_yama has joined #openstack-swift | 23:34 | |
| *** d0ugal has joined #openstack-swift | 23:36 | |
| *** hosanai has joined #openstack-swift | 23:47 | |
| *** ChanServ sets mode: +v hosanai | 23:47 | |
| *** NM has joined #openstack-swift | 23:52 | |
| *** chlong has joined #openstack-swift | 23:56 | |
| *** garthb has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!