| kota_ | morning | 00:04 |
|---|---|---|
| kota_ | tdasilva: the discussion place for pyeclib/liberasurecode issues moved to somewhere (lanunchpad) too? | 00:06 |
| kota_ | tdasilva: for them, we have sitll a bunch of open issues like here, https://bitbucket.org/kmgreen2/pyeclib/issues?status=new&status=open | 00:06 |
| *** bill_az_ has quit IRC | 00:17 | |
| *** lyrrad has quit IRC | 00:17 | |
| *** Jeffrey4l has joined #openstack-swift | 00:17 | |
| *** itlinux has joined #openstack-swift | 00:18 | |
| *** arch-nemesis has quit IRC | 00:21 | |
| *** Vinsh has quit IRC | 00:24 | |
| *** bill_az_ has joined #openstack-swift | 00:26 | |
| *** klamath has quit IRC | 00:26 | |
| *** klamath has joined #openstack-swift | 00:26 | |
| *** klamath has quit IRC | 00:27 | |
| *** klamath has joined #openstack-swift | 00:28 | |
| *** Suyi has quit IRC | 00:37 | |
| *** kei_yama has quit IRC | 00:53 | |
| *** kei_yama has joined #openstack-swift | 00:53 | |
| *** itlinux has quit IRC | 00:57 | |
| *** itlinux has joined #openstack-swift | 00:58 | |
| *** gyee has quit IRC | 01:03 | |
| *** itlinux has quit IRC | 01:03 | |
| openstackgerrit | Tim Burke proposed openstack/swift: Buffer DiskfileWriter writes https://review.openstack.org/91133 | 01:09 |
| clayg | kota_: i'm with you bro - where do I file issues against openstack/pyeclib now? the new 1.2.1 release is broken w/o a liberasure release :D | 01:09 |
| *** asettle has joined #openstack-swift | 01:10 | |
| kota_ | Hi clayg, I'm not sure where we should discuss the issue. Probably bitbucket's issue list is still working for tsg- and Kevin. | 01:14 |
| *** esker has joined #openstack-swift | 01:15 | |
| clayg | well if we're going to be maintaining them in openstack/ namespace we should probably have issues there? maybe luanchpad projects? | 01:15 |
| clayg | https://launchpad.net/pyeclib !! | 01:15 |
| kota_ | looks like tdasilva has been working to create https://launchpad.net/pyeclib | 01:15 |
| kota_ | clayg: bro! | 01:15 |
| kota_ | I found liberasurecode version, https://launchpad.net/liberasurecode | 01:16 |
| kota_ | too | 01:16 |
| *** asettle has quit IRC | 01:17 | |
| clayg | openstack is magic | 01:18 |
| clayg | tdasilva: https://bugs.launchpad.net/pyeclib/+bug/1586220 | 01:18 |
| openstack | Launchpad bug 1586220 in PyECLib "ImportError: /usr/lib/python2.7/dist-packages/pyeclib_c.so: undefined symbol: liberasurecode_backend_available" [Undecided,New] | 01:18 |
| kota_ | either is fine to me becuase I can get subscription from bitbucket too. But for others working on openstack, launchpad is probably good place to discuss issues. | 01:18 |
| kota_ | clayg: nice | 01:19 |
| *** esker has quit IRC | 01:19 | |
| clayg | kota_: if you have any issues you're *actively* tracking upstream it'd be good I think to get the migrated over to launchpad (manually?) you can reference the bitbucket issue - but the plan is that openstack/ is the new home and we'll maintain those libraries via gerrit | 01:20 |
| clayg | which is great! | 01:20 |
| kota_ | yeah | 01:22 |
| *** diogogmt has quit IRC | 01:25 | |
| *** mingyu has joined #openstack-swift | 01:34 | |
| *** bill_az_ has quit IRC | 01:36 | |
| *** mingyu has quit IRC | 01:38 | |
| *** kei_yama has quit IRC | 01:42 | |
| *** kei_yama has joined #openstack-swift | 01:42 | |
| *** esker has joined #openstack-swift | 01:51 | |
| *** janonymous_ has quit IRC | 01:53 | |
| *** esker has quit IRC | 01:55 | |
| *** itlinux has joined #openstack-swift | 01:59 | |
| *** itlinux has quit IRC | 02:04 | |
| *** mingyu has joined #openstack-swift | 02:16 | |
| *** nadeem has joined #openstack-swift | 02:19 | |
| *** vint_bra has joined #openstack-swift | 02:23 | |
| *** openstackgerrit has quit IRC | 02:36 | |
| *** openstackgerrit has joined #openstack-swift | 02:43 | |
| *** arch-nemesis has joined #openstack-swift | 02:54 | |
| *** openstackgerrit has quit IRC | 02:56 | |
| *** nadeem has quit IRC | 02:58 | |
| *** jamielennox is now known as jamielennox|away | 03:01 | |
| -openstackstatus- NOTICE: Gerrit is going offline briefly to check possible filesystem corruption | 03:02 | |
| *** ChanServ changes topic to "Gerrit is going offline briefly to check possible filesystem corruption" | 03:02 | |
| *** sams-gleb has joined #openstack-swift | 03:07 | |
| *** sams-gleb has quit IRC | 03:12 | |
| mattoliverau | notmyname, clayg: grrr, sqlite OFFSET turns out not to be the greatest. Not on really large containers anyway. But think I've found a solution that's super quick, just maybe not always pick the best. I decided to blog it (I know haven't touched my blog in 3 years) so others don't have to find out the hard way like me: http://oliver.net.au/?p=267 | 03:19 |
| *** openstackgerrit has joined #openstack-swift | 03:19 | |
| *** dmorita has quit IRC | 03:19 | |
| *** sheel has joined #openstack-swift | 03:19 | |
| *** dmorita has joined #openstack-swift | 03:22 | |
| *** ChanServ changes topic to "Let's talk! We're nice | Review Dashboard: https://goo.gl/mtEv1C | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/ | Meetings: https://wiki.openstack.org/wiki/Meetings/Swift | Community Dashboard: http://not.mn/swift/swift_community_dashboard.html" | 03:22 | |
| -openstackstatus- NOTICE: after a quick check, gerrit and its filesystem have been brought back online and should be working again | 03:22 | |
| *** mvk_ has joined #openstack-swift | 03:32 | |
| openstackgerrit | Janie Richling proposed openstack/swift: crypto - Don't use 'meta' for crypto-meta dict https://review.openstack.org/321937 | 03:33 |
| notmyname | mattoliverau: interesting. nice update | 03:35 |
| *** mkrcmari__ has quit IRC | 03:36 | |
| mattoliverau | ignore my bad english.. or should I say Australian. | 03:36 |
| notmyname | I've long ago stopped thinking it was "English" ;-) | 03:37 |
| *** dmorita has quit IRC | 03:37 | |
| *** dmorita has joined #openstack-swift | 03:38 | |
| mattoliverau | notmyname: :P | 03:40 |
| mattoliverau | also izip_longest makes the code sample _so_ much shorter. | 03:40 |
| mattoliverau | damn fancy iterators that make life easier :P | 03:41 |
| * mattoliverau goes to find lunch | 03:41 | |
| clayg | mattoliverau: yeah it has to *acctually* order all of them to find whos in that posistion - interesting | 03:41 |
| *** dmorita has quit IRC | 03:41 | |
| mattoliverau | so I could use the OFFSET on smaller containers to get it "right" and the fuzzy apporach on large.. or just do the latter for everything. | 03:42 |
| mattoliverau | but let me run the code on the large container again and find the next bottleneck first :P | 03:43 |
| *** links has joined #openstack-swift | 03:46 | |
| clayg | never to two different things if the one thing works | 03:49 |
| clayg | mattoliverau: i doesn't make any sense tho - there has to be a way to just ask the damn index what it has in it's middle? | 04:00 |
| clayg | this was great btw - sqlite is so great - https://www.sqlite.org/queryplanner.html | 04:02 |
| *** klrmn has quit IRC | 04:04 | |
| *** nadeem has joined #openstack-swift | 04:09 | |
| mattoliverau | clayg: maybe with a patch, but the index is a linked list (as far as I could determine) so I don't think it can jump to the middle. Tho am happy to be wrong cause it'll make my life easier. Maybe I need to look at the codes :) | 04:18 |
| clayg | mattoliverau: wait... can you do listings on the container? like marker queries and stuff - or the whole thing is just hozed? | 04:20 |
| clayg | like i was sorta thinking the problem had to be in the sorter - it's trying to get the entire 700M rows in order before thinking about the limit part | 04:22 |
| mattoliverau | Well it's running of rusty metal, so it seems hozed to me. Me thinks the split may still take a while. /me needs to play with SSDs next. | 04:22 |
| clayg | hrmm.. | 04:22 |
| clayg | gl | 04:22 |
| mattoliverau | I tried without sorting, IE no order by name | 04:23 |
| *** psachin has joined #openstack-swift | 04:24 | |
| mattoliverau | clayg: on sqllite I can LIMIT so should be able to container listing. it's more the OFFSET 350M is kinda slow :) | 04:26 |
| *** adu has joined #openstack-swift | 04:37 | |
| mahatic | jrichli: are you around? | 04:39 |
| jrichli | mahatic: yes, for a few minutes more. what's up? | 04:40 |
| *** arch-nemesis has quit IRC | 04:41 | |
| mahatic | jrichli: just had a query on trello card. but i think figured (left a comment): https://trello.com/c/v5xkmidJ/111-use-derived-iv-for-container-listing-etag-encryption-i-e-same-as-object-sysmeta-value | 04:41 |
| mahatic | I think I figured* | 04:41 |
| *** dmorita has joined #openstack-swift | 04:42 | |
| *** vint_bra has quit IRC | 04:43 | |
| jrichli | mahatic: I think the card is to change the iv used for all etag encryption to always be the one used for the container listing. currently, we use a random iv for the 'regular' object etag. | 04:44 |
| jrichli | is that what you are thinking too? | 04:44 |
| *** itlinux has joined #openstack-swift | 04:45 | |
| mahatic | jrichli: it's the other way round i suppose. the object etag currently uses derived iv and container listing etag uses a random iv. Now we want container listing etag to use derived iv as well | 04:45 |
| *** ChubYann has quit IRC | 04:46 | |
| jrichli | mahatic: yes, right. that's what I meant ;-) | 04:46 |
| *** dmorita has quit IRC | 04:46 | |
| jrichli | I think it's my bed time now! | 04:46 |
| mahatic | heh. yeah ;) | 04:47 |
| jrichli | mahatic: have a good day | 04:48 |
| mahatic | jrichli: thanks, good night! | 04:48 |
| *** itlinux has quit IRC | 04:48 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:51 | |
| *** ppai has joined #openstack-swift | 04:57 | |
| *** janonymous has joined #openstack-swift | 05:00 | |
| *** arch-nemesis has joined #openstack-swift | 05:03 | |
| *** arch-nemesis has quit IRC | 05:08 | |
| *** sams-gleb has joined #openstack-swift | 05:09 | |
| *** zaitcev has quit IRC | 05:12 | |
| *** sams-gleb has quit IRC | 05:13 | |
| *** klamath_ has joined #openstack-swift | 05:18 | |
| *** klamath has quit IRC | 05:18 | |
| *** adu has quit IRC | 05:20 | |
| *** arch-nemesis has joined #openstack-swift | 05:21 | |
| *** itlinux has joined #openstack-swift | 05:29 | |
| *** geaaru has joined #openstack-swift | 05:58 | |
| *** dmorita has joined #openstack-swift | 06:02 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift: crypto: use derived iv for container listing etag https://review.openstack.org/321970 | 06:03 |
| *** rcernin has joined #openstack-swift | 06:04 | |
| *** dmorita has quit IRC | 06:07 | |
| mahatic | acoles_: I combined this card: https://trello.com/c/v5xkmidJ/111-use-derived-iv-for-container-listing-etag-encryption-i-e-same-as-object-sysmeta-value and https://trello.com/c/F7J1xcE0/108-include-path-iv-base-in-crypto-meta-for-listing-etag in patch 321970 | 06:09 |
| patchbot | mahatic: https://review.openstack.org/#/c/321970/ - swift (feature/crypto) - crypto: use derived iv for container listing etag | 06:09 |
| mahatic | hope it's okay | 06:09 |
| *** rcernin has quit IRC | 06:15 | |
| *** rcernin has joined #openstack-swift | 06:20 | |
| *** daemontool has joined #openstack-swift | 06:44 | |
| *** hseipp has joined #openstack-swift | 06:50 | |
| *** itlinux has quit IRC | 06:51 | |
| *** sams-gleb has joined #openstack-swift | 06:56 | |
| *** hseipp has quit IRC | 06:56 | |
| *** nadeem has quit IRC | 07:04 | |
| *** tdasilva has quit IRC | 07:07 | |
| *** nadeem has joined #openstack-swift | 07:10 | |
| -openstackstatus- NOTICE: zuul required a restart due to network outages. If your change is not listed on http://status.openstack.org/zuul/ and is missing results, please issue a 'recheck'. | 07:14 | |
| *** nadeem has quit IRC | 07:22 | |
| *** alex_osso has joined #openstack-swift | 07:25 | |
| *** tdasilva has joined #openstack-swift | 07:27 | |
| *** ChanServ sets mode: +v tdasilva | 07:27 | |
| *** daemontool has quit IRC | 07:32 | |
| *** tesseract has joined #openstack-swift | 07:36 | |
| *** mmcardle has joined #openstack-swift | 08:04 | |
| *** mmcardle1 has joined #openstack-swift | 08:09 | |
| *** mmcardle has quit IRC | 08:11 | |
| *** jordanP has joined #openstack-swift | 08:13 | |
| *** asettle has joined #openstack-swift | 08:25 | |
| *** asettle has quit IRC | 08:26 | |
| *** joeljwright has joined #openstack-swift | 08:31 | |
| *** ChanServ sets mode: +v joeljwright | 08:31 | |
| *** sycoder has joined #openstack-swift | 08:37 | |
| *** sanchitmalhotra has quit IRC | 08:38 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift: crypto: use derived iv for container listing etag https://review.openstack.org/321970 | 08:42 |
| *** dmorita has joined #openstack-swift | 08:42 | |
| *** acoles_ is now known as acoles | 08:43 | |
| *** dmk0202 has joined #openstack-swift | 08:46 | |
| *** dmorita has quit IRC | 08:47 | |
| sycoder | hi...I am having an 503 error in my swift installation, does anyone have some advice on what could be the reasons/solution ? | 08:48 |
| *** mingyu has quit IRC | 08:50 | |
| *** ouchkernel has quit IRC | 09:02 | |
| *** ouchkernel has joined #openstack-swift | 09:06 | |
| *** jmccarthy has joined #openstack-swift | 09:12 | |
| *** jmccarthy has quit IRC | 09:15 | |
| *** daemontool has joined #openstack-swift | 09:18 | |
| *** jmccarthy has joined #openstack-swift | 09:20 | |
| *** sycoder has quit IRC | 09:39 | |
| *** SkyRocknRoll has quit IRC | 09:40 | |
| *** permalac has joined #openstack-swift | 09:54 | |
| *** permalac has quit IRC | 09:54 | |
| *** permalac has joined #openstack-swift | 09:54 | |
| *** permalac has quit IRC | 09:55 | |
| *** permalac has joined #openstack-swift | 09:55 | |
| *** SkyRocknRoll has joined #openstack-swift | 09:56 | |
| *** ouchkernel has quit IRC | 10:38 | |
| *** openstackgerrit has quit IRC | 10:47 | |
| *** openstackgerrit has joined #openstack-swift | 10:48 | |
| *** bzurawski has joined #openstack-swift | 10:49 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - remove default root secret https://review.openstack.org/318208 | 11:26 |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - Don't use 'meta' for crypto-meta dict https://review.openstack.org/321937 | 11:38 |
| *** sams-gleb has quit IRC | 11:39 | |
| *** openstackgerrit has quit IRC | 11:47 | |
| *** openstackgerrit has joined #openstack-swift | 11:48 | |
| *** daemontool has quit IRC | 11:51 | |
| *** daemontool has joined #openstack-swift | 11:51 | |
| *** daemontool has quit IRC | 12:08 | |
| *** daemontool has joined #openstack-swift | 12:08 | |
| *** mmcardle1 has quit IRC | 12:09 | |
| *** sams-gleb has joined #openstack-swift | 12:12 | |
| *** daemontool has quit IRC | 12:18 | |
| *** daemontool has joined #openstack-swift | 12:18 | |
| *** dmellado is now known as dmellado|lunch | 12:20 | |
| *** dmellado|lunch is now known as dmellado | 12:20 | |
| *** itlinux has joined #openstack-swift | 12:22 | |
| *** mkrcmari__ has joined #openstack-swift | 12:23 | |
| *** mvk_ has quit IRC | 12:26 | |
| *** klamath has joined #openstack-swift | 12:28 | |
| *** klamath_ has quit IRC | 12:28 | |
| *** mkrcmari__ has quit IRC | 12:30 | |
| *** mkrcmari__ has joined #openstack-swift | 12:30 | |
| *** daemontool has quit IRC | 12:31 | |
| *** daemontool has joined #openstack-swift | 12:32 | |
| *** mkrcmari__ has quit IRC | 12:40 | |
| *** daemontool has quit IRC | 12:42 | |
| *** daemontool has joined #openstack-swift | 12:42 | |
| *** dmorita has joined #openstack-swift | 12:44 | |
| *** itlinux has quit IRC | 12:45 | |
| *** itlinux has joined #openstack-swift | 12:47 | |
| *** openstackgerrit has quit IRC | 12:48 | |
| *** dmorita has quit IRC | 12:48 | |
| *** openstackgerrit has joined #openstack-swift | 12:49 | |
| *** ppai has quit IRC | 12:49 | |
| *** klamath has quit IRC | 12:53 | |
| *** klamath has joined #openstack-swift | 12:53 | |
| *** mmcardle has joined #openstack-swift | 12:58 | |
| *** MikeG451 has joined #openstack-swift | 13:00 | |
| *** kei_yama has quit IRC | 13:06 | |
| *** mkrcmari__ has joined #openstack-swift | 13:08 | |
| *** diogogmt has joined #openstack-swift | 13:12 | |
| *** StraubTW has joined #openstack-swift | 13:13 | |
| *** asettle has joined #openstack-swift | 13:27 | |
| *** diogogmt has quit IRC | 13:28 | |
| *** itlinux has quit IRC | 13:33 | |
| *** asettle has quit IRC | 13:42 | |
| *** daemontool has quit IRC | 13:48 | |
| *** daemontool has joined #openstack-swift | 13:49 | |
| openstackgerrit | Merged openstack/swift: crypto - Don't use 'meta' for crypto-meta dict https://review.openstack.org/321937 | 13:50 |
| *** diogogmt has joined #openstack-swift | 13:56 | |
| *** daemontool has quit IRC | 14:03 | |
| *** daemontool has joined #openstack-swift | 14:04 | |
| *** openstackgerrit has quit IRC | 14:18 | |
| *** openstackgerrit has joined #openstack-swift | 14:19 | |
| joeljwright | i | 14:20 |
| joeljwright | (sorry, dropped something on my keyboard) | 14:21 |
| *** alex_osso has left #openstack-swift | 14:28 | |
| *** dmk0202 has quit IRC | 14:35 | |
| *** alex____3 has joined #openstack-swift | 14:37 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - stop using key id to derive GET path keys https://review.openstack.org/318148 | 14:41 |
| *** Lickitysplitted has joined #openstack-swift | 14:43 | |
| *** Lickitysplitted_ has quit IRC | 14:43 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - stop using key id to derive GET path keys https://review.openstack.org/318148 | 14:45 |
| *** alex____3 is now known as alex_osso | 14:47 | |
| acoles | joeljwright: o | 14:57 |
| *** SkyRocknRoll has quit IRC | 15:00 | |
| *** jistr is now known as jistr|call | 15:01 | |
| clayg | tdasilva: good morning - looks like you go tmy back on the liberasurecode?! | 15:10 |
| clayg | acoles: u | 15:11 |
| *** sams-gleb has quit IRC | 15:11 | |
| *** tesseract has quit IRC | 15:11 | |
| acoles | clayg: LOL. very good | 15:11 |
| clayg | acoles: were you thinking more like "p"? | 15:12 |
| clayg | wasn't quite sure where we were going | 15:12 |
| acoles | clayg: "i-o, i-o, it's off to work we go" | 15:13 |
| tdasilva | clayg: yep, sorry about the issue | 15:13 |
| acoles | clayg: actually i was thinking i-o-u-a - what comes next? | 15:14 |
| clayg | BEER! | 15:15 |
| clayg | tdasilva: not your fault - we'll get good at all of this eventually tho ;) | 15:16 |
| *** Jeffrey4l has quit IRC | 15:16 | |
| joeljwright | acoles: :D | 15:16 |
| tdasilva | clayg: yep...need to come up with a good strategy I guess. I'm wondering if we need to time or at least make sure new pyeclib releases work with liberasurecode releases that have already been packaged by ubuntu, rh? | 15:18 |
| tdasilva | clayg: or do we just make sure that it works with the latest liberasurecode release | 15:19 |
| clayg | tdasilva: I think that one is the good enough | 15:19 |
| clayg | tdasilva: we can't control what's packaged - we just need to make sure that good versions are available | 15:19 |
| clayg | tdasilva: I'm not super stoked about the failure mode - I think maybe soe different flags int he Makefile would have prevented me from being able to build with the older liberasure-devel package - that would have hinted to me something was busted | 15:21 |
| tdasilva | clayg: yeah, my mistake is that when I built it yesterday, I built it again liberasurecode master, so I didn't see anything | 15:23 |
| clayg | tdasilva: simple thing maybe just always cut liberasure first (since it has no depends) and pyeclib second? | 15:23 |
| clayg | tdasilva: yeah I didn't notice either way because it was just a warning :\ | 15:23 |
| tdasilva | clayg: yeah, that's what I was thinking, or at least make sure to build pyeclib against the latest libEC release | 15:23 |
| *** rcernin has quit IRC | 15:24 | |
| tdasilva | and not master :P | 15:24 |
| clayg | tdasilva: well we'll have to build it against master in development? | 15:24 |
| clayg | i'm sure we can figure it out next time - just tag a liberasurecode release - it'll be fine :P | 15:24 |
| tdasilva | clayg: but for swift, I don't think we can up the release in requirements.txt until liberasurecode packages are available | 15:25 |
| *** jistr|call is now known as jistr | 15:25 | |
| clayg | tdasilva: right | 15:25 |
| tdasilva | clayg: yeah, in dev for sure, but I mean to tag a release, you want to make sure the release works with whatever is available in libEC | 15:25 |
| clayg | tdasilva: so there's a case maybe that pyeclib should be able to be updated and support different libEC versions? :\ | 15:25 |
| tdasilva | clayg: I don't follow... | 15:26 |
| clayg | tdasilva: yeah if that's possible I think that'd be good | 15:26 |
| *** mingyu has joined #openstack-swift | 15:26 | |
| clayg | tdasilva: well you can pip install pyeclib 1.2.1 but it might not work with the libEC .so's you have installed from packages? | 15:26 |
| tdasilva | correct! it doesn't | 15:27 |
| tdasilva | :) | 15:27 |
| clayg | tdasilva: I think the fact that pyeclib compiles with those warnings about missing symbols could be fixed - then at least trying to pip install would fail? | 15:27 |
| notmyname | good morning | 15:28 |
| notmyname | about to go get on the bus, but catching up on the pyeclib/liberasurecode stuff | 15:28 |
| tdasilva | clayg: I see, yes, would need to look into that...sounds like a new bug report :) | 15:28 |
| notmyname | tdasilva: did tagging the pyeclib release automatically push something to pypi? | 15:29 |
| *** links has quit IRC | 15:29 | |
| notmyname | seems like we could take advantage of the -infra setup to test master branches together and against latest stable too. I mean, that kind of thing has been set up for a lot of the projects already | 15:31 |
| notmyname | ok, time for me to walk to the bus stop | 15:34 |
| tdasilva | notmyname: it did not push into pypi, but that was an error, which I already tracked down with help from -infra | 15:36 |
| *** mingyu has quit IRC | 15:36 | |
| tdasilva | notmyname: agree on setting up tests to test different branches | 15:37 |
| *** mingyu has joined #openstack-swift | 15:37 | |
| *** mingyu has quit IRC | 15:37 | |
| tdasilva | notmyname: will fix the pypi issue in a bit | 15:38 |
| tdasilva | *will attempt | 15:38 |
| *** jordanP has quit IRC | 15:43 | |
| *** admin6 has quit IRC | 15:45 | |
| *** pgbridge has joined #openstack-swift | 15:47 | |
| *** zaitcev has joined #openstack-swift | 15:48 | |
| *** ChanServ sets mode: +v zaitcev | 15:48 | |
| *** bzurawski has quit IRC | 15:52 | |
| *** itlinux has joined #openstack-swift | 15:52 | |
| timburke | mattoliverau: the index *must* be more than just a linked-list, otherwise things like marker queries would suck. i'm *sure* there's a tree involved; just not one that can easily spit out the Nth element. i could totally see a doubly-linked list being used when iterating through the index, though | 15:58 |
| timburke | mattoliverau: nice solution, though; the unicode conversions made me a little nervous, but i think utf-8 should be well ordered like that, unlike some other encodings (hi windows-1252!) | 15:59 |
| *** lakshmiS has joined #openstack-swift | 15:59 | |
| timburke | clayg: on "there has to be a way to just ask the damn index what it has in it's middle?" - that's actually rather non-trivial. an order-statistic tree could do it, but that's an extra layer of complexity | 16:01 |
| *** mkrcmari__ has quit IRC | 16:04 | |
| *** psachin has quit IRC | 16:04 | |
| *** itlinux has quit IRC | 16:06 | |
| acoles | notmyname: IIRC you prompted this change, does it seem an appropriate approach to you? https://review.openstack.org/#/c/318208/6/swift/common/middleware/keymaster.py | 16:08 |
| patchbot | acoles: patch 318208 - swift (feature/crypto) - crypto - remove default root secret | 16:08 |
| clayg | timburke: well the exact middle might require counting - but if it is a tree they have to do *something* about balancing right? or left? or maybe left, right, right... | 16:11 |
| *** sams-gleb has joined #openstack-swift | 16:11 | |
| *** silor has joined #openstack-swift | 16:12 | |
| tdasilva | mmm...when I try a git review on liberasurecode it tries to submit a lot of old commits, not just my most recent one...any clues? | 16:12 |
| clayg | mattoliverau: I think the fuzzy middle has a good shot of getting things wrong if the extreams are outliers :\ | 16:12 |
| clayg | mattoliverau: plus I was starting to think the middle might not even be that interesting if you're considering "object_count" the middle (because of expired/deleted rows) | 16:12 |
| clayg | mattoliverau: what about "select name from objects where deleted in (1, 0) order by name offset 400000" MMMMMMMnnnnNNNNEEEhexTTT | 16:13 |
| clarkb | tdasilva: if you rebased and all those commits changed or if you are pushing to a different branch that can happen | 16:13 |
| *** joeljwright has quit IRC | 16:15 | |
| *** sams-gleb has quit IRC | 16:15 | |
| openstackgerrit | Thiago da Silva proposed openstack/liberasurecode: Release 1.2.0 https://review.openstack.org/322257 | 16:18 |
| tdasilva | clarkb: thanks! | 16:18 |
| tdasilva | clayg: so my plan is to merge that and tag that commit to cut the release, which means master HEAD, are you ok with that? | 16:19 |
| notmyname | good morning again | 16:19 |
| clayg | tdasilva: yeah that's great! | 16:20 |
| *** dmorita has joined #openstack-swift | 16:21 | |
| notmyname | acoles: yeah, that seems like much better message | 16:21 |
| acoles | notmyname: k. I'll take that as a +1 then ;) | 16:22 |
| openstackgerrit | Thiago da Silva proposed openstack/liberasurecode: Release 1.2.0 https://review.openstack.org/322257 | 16:22 |
| notmyname | acoles: and later we can figure out something about "you didn't change it from a default"? | 16:22 |
| acoles | notmyname: actually I think the plan is to not have a default when we propose to master, we only have it there now so that we can have encryption in the pipeline for gate functional tests on the feature branch | 16:23 |
| notmyname | even better :-) | 16:24 |
| acoles | notmyname: so you MUST set a value and it MUST be base64 encoded 32 bytes or more | 16:24 |
| notmyname | yeah, that sounds great | 16:25 |
| acoles | maybe we someone can figure out a check for sufficient entropy | 16:25 |
| acoles | notmyname: we could put an example of how to generate a key in doc (openssl rand -base64 32) - IDK if that is too prescriptive?? | 16:26 |
| notmyname | we probably should, as long as it's vetted by some crypto person :-) | 16:27 |
| notmyname | probably with some words around it like "make sure /dev/random has entropy" or something | 16:27 |
| acoles | k, will do. we already have some words in there from a crypto expert about high entropy. | 16:28 |
| notmyname | I saw something at LCA this year that's a tiny USB device designed specifically to product 1MB/sec of high quality random noise. basically, plug it in, and /dev/random is immediately good to go | 16:28 |
| acoles | notmyname: was it called "line manager" :P | 16:29 |
| notmyname | :-) | 16:30 |
| notmyname | speaking of management, this was an interesting read this morning http://randsinrepose.com/archives/bored-people-quit/ | 16:31 |
| notmyname | ah, here it is http://altusmetrum.org/ChaosKey/. produces 12Mb/s of random bits | 16:35 |
| notmyname | fully open hardware and software (including firmware) | 16:35 |
| *** geaaru_ has joined #openstack-swift | 16:39 | |
| *** mingyu has joined #openstack-swift | 16:39 | |
| *** geaaru has quit IRC | 16:39 | |
| *** asettle has joined #openstack-swift | 16:44 | |
| acoles | notmyname: I just updated priority reviews with a list of 6 remaining patches that would ideally merge on feature/crypto before we move to feature/crypto-review. There are a few others that imho could just follow-on on crypto-review or in case of doc carry across for continued review. | 16:45 |
| *** mmcardle has quit IRC | 16:45 | |
| notmyname | great, thanks | 16:45 |
| *** nadeem has joined #openstack-swift | 16:46 | |
| *** nadeem has quit IRC | 16:46 | |
| acoles | I'd like to start preparing the short chain of patches for crypto-review, so could you request that branch from infra? no rush - I am out on Monday anyway. | 16:46 |
| *** geaaru__ has joined #openstack-swift | 16:47 | |
| notmyname | me too (out on monday) | 16:47 |
| *** can8dnSix has joined #openstack-swift | 16:47 | |
| *** geaaru_ has quit IRC | 16:47 | |
| *** asettle has quit IRC | 16:50 | |
| *** nadeem has joined #openstack-swift | 16:52 | |
| acoles | mahatic: jrichli timburke tdasilva - see few lines above re updated priority reviews for crypto - we should focus on finishing those patches in order to move to crypto-review | 16:52 |
| tdasilva | acoles: ack | 16:53 |
| *** klrmn has joined #openstack-swift | 16:56 | |
| *** asettle has joined #openstack-swift | 16:59 | |
| *** asettle has quit IRC | 17:02 | |
| *** gyee has joined #openstack-swift | 17:05 | |
| *** SkyRocknRoll has joined #openstack-swift | 17:07 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - add overview doc (WIP) https://review.openstack.org/312096 | 17:17 |
| *** gyee has quit IRC | 17:18 | |
| *** gyee has joined #openstack-swift | 17:20 | |
| *** acoles is now known as acoles_ | 17:27 | |
| *** tqtran has joined #openstack-swift | 17:27 | |
| openstackgerrit | Thiago da Silva proposed openstack/liberasurecode: Release 1.2.0 https://review.openstack.org/322257 | 17:39 |
| *** links has joined #openstack-swift | 17:49 | |
| openstackgerrit | Thiago da Silva proposed openstack/liberasurecode: Release 1.2.0 https://review.openstack.org/322257 | 18:04 |
| openstackgerrit | Merged openstack/liberasurecode: Release 1.2.0 https://review.openstack.org/322257 | 18:10 |
| *** links has quit IRC | 18:10 | |
| tdasilva | crap, something went wrong...one sec | 18:13 |
| *** nadeem has quit IRC | 18:14 | |
| openstackgerrit | Thiago da Silva proposed openstack/liberasurecode: Add NEWS and README back https://review.openstack.org/322293 | 18:27 |
| tdasilva | timburke: ^^^ :/ | 18:27 |
| * timburke sighs | 18:29 | |
| timur | tdasilva: clayg sorry I've been MIA around the packaging stuff. I agree with Clay that at the very least pyeclib should break the build on undefined symbols. I'm juggling a few balls in the air at the moment, but I'd like to add a patch to fix that (unless someone beats me to it). The other thing that would be nice to add is a version check for the features with an ifdef (checking the version from | 18:29 |
| *** ChubYann has joined #openstack-swift | 18:29 | |
| timur | erasurecode_version.h). That'd be second patch I'd add... hopefully soon | 18:29 |
| timur | oh, hey, I can also +2 tdasilva's patch :D | 18:29 |
| tdasilva | timur: yes you can :D | 18:29 |
| openstackgerrit | Merged openstack/liberasurecode: Add NEWS and README back https://review.openstack.org/322293 | 18:30 |
| *** thumpba has joined #openstack-swift | 18:40 | |
| zaitcev | The RPM philosophy is to control dependencies with Requires: in the spec. So in theory if I write the spec right, liberasurecode always builds without error and in a repeatable fashion. | 18:44 |
| notmyname | tdasilva: I was out of the office yesterday. ping me if you need something with pyeclib or liberasurecode | 18:44 |
| *** silor has quit IRC | 19:00 | |
| *** MikeG451 has quit IRC | 19:02 | |
| tdasilva | notmyname: I just tagged a new libEC release. One of the next things I'd like to figure out is what to do with issues in bitbucket. Not sure if there's an automated way to migrate them to launchpad, so need to look into that | 19:03 |
| notmyname | tdasilva: how many are we talking about? | 19:04 |
| tdasilva | pyeclib alone has 14, which doesn't sound like a lot, but there's a ton of conversation within each of them | 19:04 |
| tdasilva | as people are trying to figure out problems | 19:04 |
| notmyname | hmmm | 19:04 |
| tdasilva | e.g: https://bitbucket.org/kmgreen2/pyeclib/issues/80/decode-segfault-with-certain-corrupted | 19:05 |
| notmyname | let me think about it. I think we should have some plan there, but it's not something that is a blocker for starting to use it | 19:05 |
| *** mmcardle has joined #openstack-swift | 19:08 | |
| *** mingyu has quit IRC | 19:08 | |
| *** itlinux has joined #openstack-swift | 19:09 | |
| tdasilva | notmyname: agreed | 19:16 |
| tdasilva | tdasilva: another item is getting the gate jobs running | 19:16 |
| *** mmcardle has quit IRC | 19:18 | |
| *** daemontool has quit IRC | 19:18 | |
| *** itlinux has quit IRC | 19:22 | |
| *** mingyu has joined #openstack-swift | 19:30 | |
| *** thumpba_ has joined #openstack-swift | 19:33 | |
| *** thumpba has quit IRC | 19:35 | |
| *** openstack has joined #openstack-swift | 21:44 | |
| clayg | maybe squashing could make a guess at left or right - solve the issue if the far left gets small after rows expire... but there's some hope it might just all eventually expire | 21:44 |
| clayg | still going to be hard to co-ordinate piviot points :\ | 21:45 |
| notmyname | if the initial shard method is "take X rows from the top, repeat", then we could do a binary method on the smaller ones. but I'm not sure the top/prefix method wouldn't end up with the same result | 21:45 |
| clayg | notmyname: if we can get by with only one thing that works I think that's best :) | 21:45 |
| notmyname | oh yeah. coordinating the pivots | 21:45 |
| notmyname | yeah. one thing >>> two things | 21:46 |
| *** dmorita has joined #openstack-swift | 21:46 | |
| clayg | even in the quorm situation you have the issues with handoffs and over replication - but as long as you have a consistent reconciliation maybe it'll all work out eventually | 21:46 |
| *** esker has quit IRC | 21:47 | |
| clayg | I'd love it if we could figure out some consistent naming such that even if two split brains come up with a different row to split on they still start populating the same named shard - then those guys can work out row reconciliation on mostly independently if needed | 21:48 |
| *** thumpba has quit IRC | 21:48 | |
| clayg | no good ideas at the moment tho :'( | 21:49 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!