| *** tsg has joined #openstack-swift | 00:03 | |
| *** cds has quit IRC | 00:11 | |
| *** openstackstatus has quit IRC | 00:23 | |
| *** openstackstatus has joined #openstack-swift | 00:24 | |
| *** ChanServ sets mode: +v openstackstatus | 00:24 | |
| *** kyles_ne has quit IRC | 00:25 | |
| *** kyles_ne has joined #openstack-swift | 00:26 | |
| *** kyles_ne has quit IRC | 00:30 | |
| *** dmorita has joined #openstack-swift | 00:34 | |
| *** elambert has quit IRC | 00:48 | |
| *** shri has quit IRC | 00:49 | |
| *** haomaiwang has quit IRC | 00:52 | |
| *** TaiSHi has quit IRC | 01:06 | |
| *** TaiSHi has joined #openstack-swift | 01:09 | |
| *** TaiSHi has quit IRC | 01:09 | |
| *** TaiSHi has joined #openstack-swift | 01:09 | |
| *** themadcanudist has left #openstack-swift | 01:11 | |
| *** kota_ has joined #openstack-swift | 01:16 | |
| *** haomaiwang has joined #openstack-swift | 01:19 | |
| *** hhuang has joined #openstack-swift | 01:33 | |
| *** haomai___ has joined #openstack-swift | 01:39 | |
| *** haomaiwang has quit IRC | 01:42 | |
| *** oomichi_ has joined #openstack-swift | 01:44 | |
| *** haomai___ has quit IRC | 01:55 | |
| *** haomaiwang has joined #openstack-swift | 01:56 | |
| *** haomaiw__ has joined #openstack-swift | 02:01 | |
| *** haomaiwang has quit IRC | 02:05 | |
| *** nosnos has joined #openstack-swift | 02:06 | |
| *** openstackgerrit has quit IRC | 02:45 | |
| *** hhuang has quit IRC | 02:46 | |
| *** mahatic has quit IRC | 03:01 | |
| *** elambert has joined #openstack-swift | 03:04 | |
| *** dmsimard is now known as dmsimard_away | 03:15 | |
| *** fbo has quit IRC | 03:19 | |
| *** fbo has joined #openstack-swift | 03:28 | |
| *** JoshuaKim has joined #openstack-swift | 03:29 | |
| JoshuaKim | Hi, All I have a question about installing swift production mode. | 03:29 |
|---|---|---|
| JoshuaKim | Is there any document about installing production mode? | 03:30 |
| *** JoshuaKim has quit IRC | 03:30 | |
| *** haomaiw__ has quit IRC | 03:34 | |
| *** haomaiwang has joined #openstack-swift | 03:34 | |
| *** nosnos has quit IRC | 03:35 | |
| *** haomai___ has joined #openstack-swift | 03:43 | |
| *** haomaiwang has quit IRC | 03:46 | |
| *** tsg has quit IRC | 04:00 | |
| *** tsg has joined #openstack-swift | 04:20 | |
| zaitcev | Use the multinode install doc. | 04:24 |
| zaitcev | There's an updated version it at docs.opentack.org. | 04:24 |
| *** JoshuaKim has joined #openstack-swift | 04:25 | |
| *** JoshuaKim has quit IRC | 04:26 | |
| *** nosnos has joined #openstack-swift | 04:35 | |
| *** tsg has quit IRC | 04:37 | |
| *** zaitcev has quit IRC | 05:01 | |
| *** occup4nt has joined #openstack-swift | 05:06 | |
| *** occupant has quit IRC | 05:09 | |
| *** tsg has joined #openstack-swift | 05:24 | |
| *** elambert has quit IRC | 05:28 | |
| *** kopparam has joined #openstack-swift | 05:36 | |
| *** ttrumm has joined #openstack-swift | 06:03 | |
| *** tsg has quit IRC | 06:17 | |
| mattoliverau | I'm calling it a day, night all. | 06:28 |
| *** nshaikh has joined #openstack-swift | 06:50 | |
| *** k4n0 has joined #openstack-swift | 06:56 | |
| *** nshaikh has quit IRC | 07:02 | |
| *** k4n0 has quit IRC | 07:02 | |
| *** mrsnivvel has quit IRC | 07:12 | |
| *** acoles_away is now known as acoles | 07:14 | |
| *** k4n0 has joined #openstack-swift | 07:14 | |
| *** jistr has joined #openstack-swift | 07:23 | |
| *** geaaru has joined #openstack-swift | 07:24 | |
| *** mrsnivvel has joined #openstack-swift | 07:25 | |
| acoles | mahatic: when your test fails with this output http://paste.openstack.org/show/119726/ then you are seeing this bug https://bugs.launchpad.net/python-swiftclient/+bug/1376878 | 07:25 |
| acoles | mahatic: its a race, so may not always occur. I have a patch to fix this in review https://review.openstack.org/126515, so you could rebase yours on that, or just hope for the best:) ping me here if you need help to rebase. | 07:30 |
| acoles | mahatic: it being a race may explain why you and notmyname were seeing different result, if you were (i'm just looking at scrollback here) | 07:32 |
| *** haomai___ has quit IRC | 07:33 | |
| *** haomaiwa_ has joined #openstack-swift | 07:34 | |
| *** haomaiw__ has joined #openstack-swift | 07:35 | |
| *** haomaiwa_ has quit IRC | 07:39 | |
| *** k4n0 has quit IRC | 07:42 | |
| *** tab___ has joined #openstack-swift | 07:42 | |
| *** k4n0 has joined #openstack-swift | 07:55 | |
| *** jistr has quit IRC | 08:12 | |
| *** nellysmitt has joined #openstack-swift | 08:24 | |
| *** oomichi_ has quit IRC | 08:24 | |
| *** jistr has joined #openstack-swift | 08:31 | |
| *** SkyRocknRoll has joined #openstack-swift | 08:59 | |
| *** SkyRocknRoll has joined #openstack-swift | 08:59 | |
| *** nshaikh has joined #openstack-swift | 09:01 | |
| *** kota_ has quit IRC | 09:13 | |
| *** haomaiw__ has quit IRC | 09:19 | |
| *** X019 has joined #openstack-swift | 09:19 | |
| *** haomaiwa_ has joined #openstack-swift | 09:19 | |
| *** kopparam has quit IRC | 09:27 | |
| *** haomaiwa_ has quit IRC | 09:34 | |
| *** haomai___ has joined #openstack-swift | 09:36 | |
| *** kopparam has joined #openstack-swift | 09:36 | |
| *** exploreshaifali has joined #openstack-swift | 09:42 | |
| *** JoshuaKim has joined #openstack-swift | 09:43 | |
| *** kopparam has quit IRC | 09:50 | |
| *** nshaikh has left #openstack-swift | 09:51 | |
| *** haomai___ has quit IRC | 09:53 | |
| *** kopparam has joined #openstack-swift | 09:54 | |
| *** haomaiwang has joined #openstack-swift | 09:54 | |
| *** haomaiw__ has joined #openstack-swift | 09:56 | |
| *** haomaiwang has quit IRC | 09:59 | |
| *** JoshuaKim has quit IRC | 10:03 | |
| *** dmorita has quit IRC | 10:05 | |
| *** JoshuaKim has joined #openstack-swift | 10:05 | |
| *** X019 has quit IRC | 10:07 | |
| *** exploreshaifali has quit IRC | 10:07 | |
| *** kopparam has quit IRC | 10:20 | |
| *** kopparam has joined #openstack-swift | 10:21 | |
| *** X019 has joined #openstack-swift | 10:23 | |
| *** kopparam has quit IRC | 10:26 | |
| *** mkerrin has quit IRC | 10:29 | |
| *** mkerrin has joined #openstack-swift | 10:37 | |
| *** aix_ has quit IRC | 10:38 | |
| *** X019 has quit IRC | 10:39 | |
| *** haomaiw__ has quit IRC | 10:39 | |
| *** haomaiwang has joined #openstack-swift | 10:39 | |
| *** kopparam has joined #openstack-swift | 10:40 | |
| *** haomaiw__ has joined #openstack-swift | 10:57 | |
| *** haomaiwang has quit IRC | 10:58 | |
| *** aix_ has joined #openstack-swift | 11:07 | |
| *** dmsimard_away is now known as dmsimard | 11:07 | |
| *** kopparam has quit IRC | 11:11 | |
| *** fbo has quit IRC | 11:30 | |
| *** fbo has joined #openstack-swift | 11:33 | |
| *** JoshuaKim has quit IRC | 11:34 | |
| *** mahatic has joined #openstack-swift | 11:38 | |
| *** mahatic_ has joined #openstack-swift | 11:50 | |
| *** StevenK has quit IRC | 11:50 | |
| *** StevenK has joined #openstack-swift | 11:51 | |
| mahatic_ | cschwede, hi, around? | 11:51 |
| *** mahatic has quit IRC | 11:52 | |
| *** aix_ has quit IRC | 11:59 | |
| cschwede | mahatic_: yes :) | 12:02 |
| cschwede | mahatic_: how can i help? | 12:03 |
| mahatic_ | cschwede, great :) If I use "int(options.get('segment_size', 0))", it throws a TypeError | 12:04 |
| mahatic_ | cschwede, for this patch https://review.openstack.org/#/c/125275/ | 12:04 |
| mahatic_ | cschwede, sorry, i forgot to update you on this yesterday | 12:04 |
| mahatic_ | cschwede, so i'm sticking to the current check | 12:04 |
| cschwede | mahatic_: let me have a look | 12:04 |
| mahatic_ | cschwede, okay | 12:04 |
| acoles | cschwede: if you have time please review this https://review.openstack.org/127196 , it is causing test failures including on mahatic_ patch | 12:08 |
| acoles | cschwede: there is a dependant patch with a test, i'm not sure the test should be erged (?). thanks | 12:09 |
| *** JoshuaKim has joined #openstack-swift | 12:09 | |
| acoles | merged | 12:09 |
| JoshuaKim | I have a question about swift-recon, any document for using swift-recon?? | 12:11 |
| cschwede | mahatic_: yes, you’re right, because the default in shell.py is None, not 0. I updated my comment on the review | 12:11 |
| cschwede | acoles: sure | 12:11 |
| mahatic_ | cschwede, acoles, something really weird, after the changes, these tests "sh run_tests.sh" and "./.unittests", sometimes pass, sometimes fail. | 12:11 |
| cschwede | mahatic_: with a rebase on top of acoles patches? | 12:12 |
| mahatic_ | acoles, is this behavior due to those bugs you have pointed? | 12:12 |
| mahatic_ | cschwede, ah nope. will do now | 12:12 |
| mahatic_ | cschwede, acoles this one has changes still pending, correct? https://review.openstack.org/#/c/125759/ | 12:13 |
| mahatic_ | should i not be waiting for those changes to be completed? | 12:13 |
| cschwede | acoles: nice catch in https://review.openstack.org/#/c/127196/ - i needed a second to spot the behaviour. interesting! | 12:14 |
| *** mkollaro has joined #openstack-swift | 12:15 | |
| acoles | cschwede: yeas, took me a while to track it down | 12:16 |
| acoles | cschwede: the clue is 'multithreading.py' - a sure sign of trouble :) | 12:16 |
| acoles | mahatic_: there are two race conditions with fixes pending: https://review.openstack.org/#/c/127196/ and https://review.openstack.org/#/c/126515/ | 12:18 |
| mahatic_ | acoles, okay. You have pointed to this one in the comment as well: https://review.openstack.org/#/c/125759/ | 12:19 |
| mahatic_ | acoles, that needs to be merged too? | 12:19 |
| cschwede | acoles: heh, so true. „with careful Asynchronous Be operations, order get of things out may.“ | 12:19 |
| mahatic_ | acoles, and my question is, since there are changes pending from your side, should i still go ahead with a rebase? | 12:20 |
| acoles | cschwede: true so is that :) | 12:20 |
| cschwede | acoles: i’m wondering about lines 48,57 and 59 in https://review.openstack.org/#/c/125759/4/swiftclient/multithreading.py - what is the reason for the change? | 12:21 |
| *** openstackgerrit has joined #openstack-swift | 12:24 | |
| mahatic_ | cschwede, since there are changes pending, should i still go ahead with the rebase? | 12:26 |
| acoles | cschwede: so, in my tests https://review.openstack.org/#/c/125759/4/tests/unit/test_shell.py line 1058, the mock was not taking effect with the default set in the __init__ args | 12:26 |
| acoles | cschwede: i didn't understand why but that change made it work! | 12:27 |
| cschwede | mahatic_: sure; after that you can simply click rebase in Gerrit if there is a change in the dependencies | 12:27 |
| acoles | cschwede: do you know if mahatic_ 's patch can be dependent on both my race fixes? | 12:28 |
| mahatic_ | cschwede, hmm, can you help me with rebase? Because they're not merged, "git rebase" won't work i think. How do i go about it? | 12:29 |
| acoles | by cherry picking the two and the doing git review? | 12:29 |
| acoles | cschwede: ^^ | 12:29 |
| cschwede | acoles: ah, good question. i think mahati needs to rebase on of your patches first to the other | 12:30 |
| acoles | mahatic_: you need to rebase your patch and then push to gerrit. | 12:30 |
| cschwede | acoles: mahatic: one second | 12:30 |
| acoles | mahatic_: problem is there are two of my patches to rebase on, give me a minute to experiment | 12:30 |
| mahatic_ | acoles, cschwede okay, will wait | 12:31 |
| *** marcusvrn_ has joined #openstack-swift | 12:32 | |
| cschwede | acoles, mahatic_: this is what i did: http://paste.openstack.org/show/119881/ | 12:34 |
| cschwede | checked out both patches from acoles, rebased the longer one on top of the short one (because it is likely that the shorter gets merged first), fixed the conflict, and rebased mahatic_ patch on top of acoles | 12:35 |
| *** nosnos has quit IRC | 12:35 | |
| *** nosnos has joined #openstack-swift | 12:36 | |
| acoles | cschwede: mahatic_ : hold on - i may have confused things - i don't think mahatic_ needs 125759 | 12:37 |
| acoles | the two races are fixed in 127196 and... | 12:38 |
| *** erlon_ has joined #openstack-swift | 12:38 | |
| cschwede | acoles: ok, then it gets quite easy | 12:38 |
| acoles | ...and https://review.openstack.org/#/c/126515/ | 12:38 |
| *** JoshuaKim has quit IRC | 12:39 | |
| acoles | cschwede: i need to back the sysexit race fix out of 125759, i moved it to 127196 to hopefully get it merged faster | 12:39 |
| *** nosnos has quit IRC | 12:40 | |
| mahatic_ | cschwede, acoles looking at it | 12:40 |
| acoles | mahatic_: so following cschwede 's process, i did this: http://paste.openstack.org/show/119887/ | 12:44 |
| acoles | mahatic_: once you have done that then 'git review', you will be asked 'Do you really want to submit the above commits? | 12:46 |
| acoles | ', answer yes and you will create a new version of your change that depends on those two other patches | 12:46 |
| mahatic_ | acoles, okay, this might be silly but, why are there two commands, one with -d and one without? | 12:46 |
| *** JoshuaKim has joined #openstack-swift | 12:47 | |
| mahatic_ | acoles, git review 126515 and git review -d 126515 | 12:47 |
| acoles | mahatic_: agh, sorry ignore line 2854 that was my mistake | 12:47 |
| acoles | mahatic_: there are no silly questions when using git :) | 12:48 |
| mahatic_ | acoles, sure. also, what does -d mean? | 12:48 |
| *** JoshuaKim has quit IRC | 12:48 | |
| mahatic_ | acoles, heh :) | 12:48 |
| mahatic_ | description i guess | 12:50 |
| acoles | the -d <number> option will cause the gerrit review with that number to be fetched from gerrit, a new branch created and switch to that branch | 12:50 |
| cschwede | mahatic_: „git review -d 123456“ simply checks out review nr. 123456 | 12:50 |
| *** miqui has joined #openstack-swift | 12:50 | |
| cschwede | acoles: ok, your explanation is better ;) | 12:50 |
| *** aix has joined #openstack-swift | 12:50 | |
| acoles | cschwede: mahatic_ : or... it does lots of stuff :) | 12:51 |
| mahatic_ | cschwede, :) acoles :D okay, thanks! | 12:51 |
| mahatic_ | acoles, oh also, I will have to first commit my changes right? (updated code according to the comments) | 12:52 |
| acoles | mahatic_: if you have not made your changes yet, do the rebase process, then make your changes, then git commit --amend -a , then push to gerrit with git review | 12:54 |
| mahatic_ | acoles, okay | 12:55 |
| mahatic_ | acoles, i have made them, but i again quickly do them again after the rebase process | 12:55 |
| mahatic_ | i can* | 12:55 |
| *** tdasilva has joined #openstack-swift | 12:58 | |
| mahatic_ | acoles, cschwede i interchanged the sequence by mistake! i first did git review -d 126515 and then git review -d 127196 | 12:59 |
| acoles | mahatic_: if git did not complain then it does not matter | 12:59 |
| mahatic_ | acoles, hmm okay. it did not | 13:00 |
| acoles | but now swap commands at line 2856 and 2859 in the pastebin | 13:00 |
| *** CaioBrentano has joined #openstack-swift | 13:00 | |
| mahatic_ | acoles, yeah that's what i was going to ask. will do that | 13:01 |
| acoles | because you have ended up on different branch | 13:01 |
| mahatic_ | correct | 13:01 |
| acoles | btw, 'log' is my alias for 'git log' - use 'git log' to check what is happening as you go | 13:01 |
| mahatic_ | acoles, but is there a reason that i do git checkout review/mahati/125275 in between these two? | 13:02 |
| mahatic_ | acoles, yeah | 13:02 |
| acoles | mahatic_: yes, first you are on branch for one of the race fixes, and you rebase it onto the branch with the ther race fix | 13:03 |
| acoles | then you switch to branch with your patch and rebase that onto the rebased race fix | 13:04 |
| acoles | if you have your patch on a branch already the swap review/mahati/125275 for your local branch name | 13:04 |
| acoles | git checkout <local brach> | 13:05 |
| openstackgerrit | Alistair Coles proposed a change to openstack/python-swiftclient: Fix cross account upload using --os-storage-url https://review.openstack.org/125759 | 13:06 |
| mahatic_ | acoles, yes got it. but quick question: | 13:06 |
| mahatic_ | git review -d 126515-> will pick up the changes and switch to that branch | 13:07 |
| *** Sanchit has joined #openstack-swift | 13:07 | |
| mahatic_ | git review -d 127196 - > will again pick up these changes and switches to this branch | 13:07 |
| mahatic_ | git rebase review/alistair_coles/p-bug1376878-container-put-race -> I will rebase to this patch | 13:08 |
| mahatic_ | git checkout master (because im on master branch) -> will switch to my branch | 13:08 |
| mahatic_ | acoles, git rebase review/alistair_coles/p-bug1379229-sysexit-race -> now why do i do this again? | 13:08 |
| acoles | mahatic_: you have not done this last step before | 13:09 |
| acoles | mahatic_: so, taking your steps above one at a time... | 13:09 |
| acoles | git review -d 126515 creates branch review/alistair_coles/p-bug1376878-container-put-race | 13:10 |
| *** ppai has joined #openstack-swift | 13:10 | |
| mahatic_ | acoles, i have only done "git review -d 126515" and this "git review -d 127196" until now. Confirming before i do the rest so i don't screw up more :) | 13:10 |
| mahatic_ | in that sequence | 13:10 |
| acoles | git review -d 127196 creates branch review/alistair_coles/p-bug1379229-sysexit-race | 13:10 |
| Sanchit | Hi, Regarding the ACL permissions on Container, Is it possible to specify a particular role in 'X-Container-Read' and then, all the users with that particular role can access the objects in the specified container? In general terms, is role-based ACL a feature in openstack-swift? | 13:11 |
| Sanchit | clayg: Hi, Can you help me with the above query? | 13:11 |
| acoles | now you are on branch review/alistair_coles/p-bug1379229-sysexit-race | 13:11 |
| mahatic_ | acoles, okay | 13:12 |
| acoles | mahatic_: git rebase review/alistair_coles/p-bug1376878-container-put-race will now insert the one differing commit on the current branch | 13:12 |
| acoles | now you change to your branch (master, or whatever) | 13:12 |
| acoles | git rebase review/alistair_coles/p-bug1379229-sysexit-race will apply the *two* differing commits to your current branch | 13:13 |
| mahatic_ | acoles, ah okay! | 13:14 |
| *** mahatic__ has joined #openstack-swift | 13:16 | |
| *** mahatic_ has quit IRC | 13:20 | |
| mahatic__ | acoles, http://paste.openstack.org/show/119902/ i get that conflict | 13:21 |
| mahatic__ | cschwede, acoles, pretty much what i did: http://paste.openstack.org/show/119905/ | 13:33 |
| cschwede | mahatic__: no you need to edit swiftclient/service.py and look for something like „<<<<<< HEAD „ and fix that | 13:34 |
| cschwede | mahatic__: and afterwards, do a „git add swiftclient/service.py“ and then a „git rebase —continue“ | 13:34 |
| cschwede | mahatic__: and then, if you do a „git log“, you should see your patch on top of acoles | 13:35 |
| *** Nadeem has joined #openstack-swift | 13:37 | |
| mahatic__ | cschwede, ohh, should i have to remove that head and retain my changes and then do git add? | 13:37 |
| cschwede | mahatic__: one second, i create a paste | 13:38 |
| *** SkyRocknRoll has quit IRC | 13:38 | |
| mahatic__ | cschwede, okay. my changes are sandwiched in between. http://paste.openstack.org/show/119906/ Sorry, this is confusing | 13:38 |
| cschwede | mahatic__: no worries, we’re going to solve that :) | 13:39 |
| mahatic__ | cschwede, :) thanks so much | 13:39 |
| *** jokke__ is now known as jokke_ | 13:40 | |
| cschwede | mahatic__: ah, yes, your paste is exactly what i was looking for. so you need to remove line 1-10 and 18. line 1-10 is the old stuff, and git does’nt know how to fix this, thus you have to do this | 13:40 |
| cschwede | mahatic__: and when this is removed, you need to do a „git add“ | 13:40 |
| cschwede | mahatic__: effectively only leaving your new code | 13:41 |
| cschwede | mahatic__: hold on, only remove line 1, 9-10 | 13:41 |
| cschwede | mahatic__: 2-8 is ok | 13:41 |
| mahatic__ | cschwede, oh okay. Yes, i was going to ask that | 13:41 |
| mahatic__ | because i never touched it | 13:42 |
| *** tsg has joined #openstack-swift | 13:43 | |
| mahatic__ | cschwede, yup, i see my commit now. But why do i only see this one after mine: https://review.openstack.org/#/c/127196/ | 13:45 |
| mahatic__ | and not https://review.openstack.org/#/c/126515/ on too | 13:46 |
| mahatic__ | cschwede, because now 127196 has both? | 13:46 |
| openstackgerrit | Alistair Coles proposed a change to openstack/python-swiftclient: Fix race between container create jobs during upload https://review.openstack.org/126515 | 13:46 |
| openstackgerrit | Alistair Coles proposed a change to openstack/python-swiftclient: Fix cross account upload using --os-storage-url https://review.openstack.org/125759 | 13:46 |
| *** delattec has joined #openstack-swift | 13:48 | |
| openstackgerrit | Alistair Coles proposed a change to openstack/python-swiftclient: Fix race between container create jobs during upload https://review.openstack.org/126515 | 13:48 |
| openstackgerrit | Alistair Coles proposed a change to openstack/python-swiftclient: Fix cross account upload using --os-storage-url https://review.openstack.org/125759 | 13:50 |
| *** cdelatte has quit IRC | 13:50 | |
| mahatic__ | cschwede, acoles git review gives me this http://paste.openstack.org/show/119907/ | 13:52 |
| cschwede | mahatic__: hmm, that’s too much imo | 13:53 |
| acoles | cschwede: ^^ i rebased 125759 | 13:53 |
| acoles | cschwede: mahatic__ : answer 'no' ! | 13:54 |
| acoles | mahatic__: you probably need to pull master and rebase onto master too! | 13:54 |
| acoles | cschwede: mahatic__ or maybe it won't matter becase those other patches have merged. hmm | 13:55 |
| acoles | mahatic__: so lets try to do sort it out - what local branch are you working on? | 13:56 |
| mahatic__ | acoles, master | 13:56 |
| acoles | mahatic__: and do you have another branch that is tracking upstream master? (for me, that is my local master) | 13:57 |
| mahatic__ | acoles, i think my local master itself does that | 13:58 |
| acoles | mahatic__: problem is you have committed your changes on master. In future, best to switch to a new branch and make all your changes there. | 13:59 |
| mahatic__ | acoles, oh, so i can pull master easily and then rebase my local branch changes? | 13:59 |
| acoles | yes, but i'm not sure what happens if you now pull master while you have your local changes on it. | 14:00 |
| mahatic__ | acoles, and now, i will have to do a 'git pull master' and redo the whole rebase process, my changes and then commit? | 14:01 |
| acoles | mahatic__: no, i think we can avoid you repeating everything - give me min to remind myself what to do | 14:01 |
| *** ttrumm has quit IRC | 14:02 | |
| mahatic__ | acoles, okay sure. But i can do that quickly if you can confirm that will resolve this. But i don't understand why should i do git pull master in the first place? (When i'm already up to date or am i not?) | 14:03 |
| cschwede | mahatic__: do you have the commit hash from your patch before the first rebase? | 14:06 |
| mahatic__ | cschwede, checking | 14:06 |
| cschwede | the last patchset on gerrit is 9a55fd0e9bf7a0db87722d4b9e346a701665d6b9 | 14:07 |
| acoles | cschwede: i'm thinking git checkout -b new; git checkout master; git reset --hard HEAD~3; git pull origin master; then rebase new on master?? | 14:08 |
| acoles | cschwede: i'm sure you have a quicker solution :) | 14:08 |
| mahatic__ | 9a55fd0e9bf7a0db87722d4b9e346a701665d6b9 | 14:08 |
| mahatic__ | cschwede, yup, looks like that's the one | 14:09 |
| cschwede | acoles: yep, that’s one solution, and no, i don’t have a quicker one :) (though i’m always using git pull —ff-only when I’m on master) | 14:09 |
| cschwede | mahatic__: ok, that means you did no changes since the last upload to gerrit, right? | 14:09 |
| mahatic__ | cschwede, after rebase process, i did my changes, did a commit and got that error | 14:10 |
| mahatic__ | cschwede, so yeah, did not upload to gerrit after that | 14:10 |
| cschwede | mahatic__: but no change before the rebase? that would simplify things | 14:10 |
| mahatic__ | cschwede, i did, but they got overwritten during the rebase process | 14:11 |
| mahatic__ | cschwede, i mean those changes were in local (did not stage/commit) | 14:12 |
| mahatic__ | did i answer that right? | 14:12 |
| cschwede | mahatic__: i think the best is to start again with a clean master. so, you need to go back to the last commit on master that was not yours - and looking at http://paste.openstack.org/show/119907/ i think that’s at least 6-7 commits | 14:13 |
| cschwede | mahatic__: i create a paste with my recommendation, and maybe acoles can have a look at this too. one minute | 14:14 |
| mahatic__ | cschwede, hmm okay | 14:14 |
| cschwede | mahatic__: so these commands clean your master branch, and then rebase your patch on top of acoles: http://paste.openstack.org/show/119913/ | 14:17 |
| cschwede | mahatic__: and then you can modify your patch as required (i think there was a comment from notmyname) | 14:17 |
| cschwede | mahatic__: and your patch is also in a separate branch then („review/mahati/125275“) | 14:18 |
| cschwede | mahatic__: thus you can keep master clean, and on master only fetch the merged commits from upstream | 14:18 |
| cschwede | mahatic__: acoles: so the idea in http://paste.openstack.org/show/119913/ is to go back a few commits (10 to be on the safe side), and then fetch all merged commits from upstream | 14:19 |
| mahatic__ | cschwede, oh okay. sure. So after i follow your steps in paste, i will do git checkout review/mahati/125275, correct? | 14:19 |
| cschwede | mahatic__: no need to, because „git review -d 125275“ already switches to that branch | 14:20 |
| acoles | cschwede: why git review -d 125759 ? thats my 'bigger' patch not the race fixes | 14:21 |
| mahatic__ | cschwede, okay | 14:22 |
| cschwede | acoles: you mean to rebase against 127196 ? I thought 125759 fixes another race? | 14:23 |
| cschwede | mahatic__: hold on, rebase against 127196 instead of 125759 might be better | 14:25 |
| mahatic__ | cschwede, yeah, haven't done anything yet. waiting for yours and acoles confirmation | 14:26 |
| acoles | cschwede: mahatic__ i have an idea - how about i rebase 126515 on 127196, then mahatic__ only needs to rebase on 127196? and 125275 is not relevant anymore | 14:28 |
| swift_fan1 | Why is it that most of the filesystem recommendations that I see, seem to recommend *xfs* for formatting the drives on the nodes in your Swift cluster ? | 14:29 |
| acoles | cschwede: i moved one of the race fixes out of 125275 into 127196 to get it merged faster, the other race fix is 126515 | 14:29 |
| mahatic__ | acoles, why would 125275 not be relevant? I would still need that right? | 14:29 |
| cschwede | acoles: too many numbers and too less coffee on my side ;) | 14:30 |
| mahatic__ | lol | 14:30 |
| *** annegent_ has joined #openstack-swift | 14:31 | |
| mahatic__ | acoles, 125275? but that is my patch (?) | 14:31 |
| acoles | mahatic__: argh, my bad, i meant 125759 is not relevant to your patch!! sorry | 14:31 |
| mahatic__ | yes. okay! | 14:31 |
| cschwede | acoles: so master -> 127196 -> 126515 -> 125275 ? | 14:32 |
| cschwede | eh, no. master -> 126515 -> 127196 -> 125275 | 14:33 |
| cschwede | ah, whatever! i get a coffee first | 14:33 |
| acoles | cschwede: yes! | 14:33 |
| acoles | cschwede: yes to chain and to coffee :) | 14:33 |
| mahatic__ | heh :D | 14:34 |
| cschwede | ok, so acoles you rebase 127196 on top of 126515, right? and when this is done, mahatic__ can do this http://paste.openstack.org/show/119921/ | 14:36 |
| * cschwede thinks that we’re close to a nice solution | 14:36 | |
| *** charz has quit IRC | 14:37 | |
| *** X019 has joined #openstack-swift | 14:37 | |
| * mahatic__ too! | 14:37 | |
| acoles | yes | 14:38 |
| cschwede | great :) | 14:38 |
| * acoles is on a call so needs 30 mins | 14:38 | |
| mahatic__ | cool :) acoles will you let me know once you're done with your rebase? | 14:39 |
| mahatic__ | sure | 14:39 |
| *** charz has joined #openstack-swift | 14:39 | |
| acoles | mahatic__: yep. btw, its not usually this hard! | 14:39 |
| mahatic__ | acoles, i understand. But because this has dependencies, it looks so | 14:40 |
| mahatic__ | And acoles, cschwede Thanks a ton for the help!! | 14:41 |
| *** k4n0 has quit IRC | 14:42 | |
| cschwede | mahatic__: np, you’re welcome! | 14:42 |
| tdasilva | swift_fan1: the default DiskFile implementation requires an FS with xattr support and the recommendation has been xfs | 14:42 |
| *** bill_az has joined #openstack-swift | 14:46 | |
| openstackgerrit | Alistair Coles proposed a change to openstack/python-swiftclient: Fix race in shell when testing for errors to raise SysExit https://review.openstack.org/127196 | 14:49 |
| openstackgerrit | Alistair Coles proposed a change to openstack/python-swiftclient: Test to verify fix to the race when testing error_count https://review.openstack.org/127197 | 14:49 |
| acoles | mahatic__: cschwede 127196 is now rebased on 126515, so mahatic__ can rebase on 127196 only | 14:51 |
| *** NM has joined #openstack-swift | 14:51 | |
| *** annegent_ has quit IRC | 14:51 | |
| mahatic__ | acoles, cschwede okay, im doing this right away: http://paste.openstack.org/show/119921/ | 14:52 |
| cschwede | mahatic__: yes; do this on the master branch (ie a „git checkout master“ first) | 14:53 |
| *** exploreshaifali has joined #openstack-swift | 14:54 | |
| mahatic__ | cschwede, already on master | 14:54 |
| swift_fan1 | tdasilva : What do you mean by DiskFile? | 14:55 |
| tdasilva | swift_fan1: basically it is a class in the object server that writes objects to disk. checkout this blog for further info: https://swiftstack.com/blog/2014/02/04/swift-extensibility/ | 14:57 |
| mahatic__ | cschwede, http://paste.openstack.org/show/119922/ should i delete the existing one and redo git review -d 127196? | 14:58 |
| cschwede | mahatic__: no need to do, git review updates it automatically if needed | 15:00 |
| *** delatte has joined #openstack-swift | 15:00 | |
| mahatic__ | cschwede, okay | 15:00 |
| *** swift_fan has joined #openstack-swift | 15:01 | |
| *** delattec has quit IRC | 15:03 | |
| *** elambert has joined #openstack-swift | 15:04 | |
| *** swift_fan1 has quit IRC | 15:04 | |
| openstackgerrit | Mahati proposed a change to openstack/python-swiftclient: Adds a user friendly message when --segment-size is a non-integer https://review.openstack.org/125275 | 15:08 |
| *** jistr has quit IRC | 15:08 | |
| *** tsg has quit IRC | 15:09 | |
| mahatic__ | cschwede, acoles finally there! thank you so much! | 15:10 |
| *** lpabon has joined #openstack-swift | 15:11 | |
| acoles | mahatic__: yay! well done | 15:12 |
| mahatic__ | acoles, :) will wait for jenkins :D | 15:12 |
| cschwede | mahatic__: congrats :) | 15:12 |
| mahatic__ | cschwede, thanks. thanks again for the time! for acoles too. | 15:13 |
| mahatic__ | :) | 15:13 |
| *** corvus is now known as jeblair | 15:30 | |
| *** lnxnut has joined #openstack-swift | 15:31 | |
| *** exploreshaifali has quit IRC | 15:34 | |
| *** exploreshaifali has joined #openstack-swift | 15:34 | |
| *** exploreshaifali has quit IRC | 15:34 | |
| *** aerwin has joined #openstack-swift | 15:35 | |
| openstackgerrit | James Page proposed a change to openstack/swift: Adjust MAX_FILE_SIZE during test on 32 bit systems https://review.openstack.org/127030 | 15:45 |
| swift_fan | I'm not 100% sure if this is true, but I heard that cloud *object* storage is much cheaper, money per GB of storage, than cloud *block* storage. | 15:50 |
| swift_fan | Does someone know what the reasons for this are? | 15:50 |
| swift_fan | :) | 15:50 |
| chmouel | cheaper hardware :) | 15:50 |
| *** mkollaro has quit IRC | 16:02 | |
| *** kyles_ne has joined #openstack-swift | 16:03 | |
| *** mkollaro has joined #openstack-swift | 16:03 | |
| notmyname | good morning, world | 16:04 |
| swift_fan | chmouel : What ? What if you buy the exact same hardware for both systems ? | 16:04 |
| swift_fan | chmouel : Basically, given that all the hardware is equal. | 16:05 |
| swift_fan | chmouel : You can still install either cloud *object* storage, or cloud *block* storage, on the same type of hardware, right ? | 16:05 |
| swift_fan | chmouel : Nothing "special" about either of them ? | 16:05 |
| *** exploreshaifali has joined #openstack-swift | 16:06 | |
| notmyname | block storage is provisioned in finite-sized chunks (eg 1TB). object storage doesn't have a provisioning step, so you get better utilization of the entire set of hardware | 16:07 |
| *** tsg has joined #openstack-swift | 16:10 | |
| swift_fan | notmyname chmouel : What exactly do you mean by "provisioning / provisioning step" ? | 16:11 |
| swift_fan | notmyname chmouel : And I thought since you're accessing storage on the block level, in cloud block storage, you're getting better utilization of the entire set of hardware than object storage, no ? | 16:11 |
| chmouel | swift_fan: so are you trying to compare the advantages of using a block storage compared to an object storage? for which use cases? those are distinct use cases | 16:12 |
| notmyname | swift_fan: you've been around a while asking a lot of questions. while I think you've found that we're happy to help people out, I think it would help us all out if you could give us some more info on your background. what are you trying to do? what is your level of experience? | 16:13 |
| *** stevew has joined #openstack-swift | 16:15 | |
| *** stevew has left #openstack-swift | 16:15 | |
| *** Murali_ has joined #openstack-swift | 16:16 | |
| Murali_ | anybody know how to increase the storage limit for /dev/loop0 6.0G 258M 5.8G 5% /opt/stack/data/swift/drives/sdb1 (devstack) | 16:17 |
| notmyname | Murali_: do you need to keep any existing data on it? | 16:18 |
| Murali_ | no... i have too large qcow images.. | 16:19 |
| notmyname | Murali_: ok. that looks like a loopback mount. is it? | 16:19 |
| Murali_ | notmyname : 6GB swift storage will not sufficient | 16:19 |
| swift_fan | notmyname : Thanks. It's mostly out of burning curiosity. But in regards to the question about level of experience, I guess you can say that I'm a hobbyist. | 16:20 |
| Murali_ | notmyname : do you know how to increase swift storage limit... or any configuration avilable ? | 16:20 |
| swift_fan | chmouel notmyname : I've got to step out for a bit, but if there's anything that you want to post, I'll be sure to read them. | 16:20 |
| glange | http://www.wikipedia.org/ | 16:22 |
| *** siwsky has joined #openstack-swift | 16:22 | |
| chmouel | Murali_: you want to do that in devstack it seems, just make sure to increase this variable https://github.com/openstack-dev/devstack/blob/master/lib/swift#L68 to something larger | 16:22 |
| chmouel | Murali_: in your local.conf :) | 16:22 |
| chmouel | sorry the smiley was for glange :) | 16:22 |
| peluse | /me runs off to read wikipedia | 16:23 |
| tdasilva | lol | 16:23 |
| Murali_ | chmouel: i see now | 16:24 |
| Murali_ | chmouel: thanks :) | 16:24 |
| chmouel | Murali_: you welcome | 16:24 |
| swift_fan | glange : wikipedia doesn't seem to have all of this knowledge..... | 16:24 |
| Murali_ | chmouel: so i just need to add SWIFT_LOOPBACK_DISK_SIZE_DEFAULT= 50 GB something like this | 16:25 |
| chmouel | Murali_: yes in your local.conf | 16:25 |
| Murali_ | ok | 16:25 |
| chmouel | Murali_: don't put spaces tho for shell variable SWIFT_LOOPBACK_DISK_SIZE_DEFAULT=50G would be the syntax | 16:26 |
| Murali_ | chmouel: ok got it....you made my day... | 16:27 |
| Murali_ | i going to try now | 16:27 |
| *** mkollaro has quit IRC | 16:37 | |
| openstackgerrit | Prashanth Pai proposed a change to openstack/swift: Fix unit tests failing in some cases https://review.openstack.org/127285 | 16:38 |
| openstackgerrit | Alistair Coles proposed a change to openstack/python-swiftclient: Fix cross account upload using --os-storage-url https://review.openstack.org/125759 | 16:49 |
| *** gyee has quit IRC | 17:01 | |
| *** tsg has quit IRC | 17:11 | |
| *** tsg has joined #openstack-swift | 17:11 | |
| *** shri has joined #openstack-swift | 17:13 | |
| *** gyee has joined #openstack-swift | 17:15 | |
| *** acoles is now known as acoles_away | 17:30 | |
| *** aerwin has quit IRC | 17:32 | |
| cschwede | glange: nah, wikipedia is biased. heard they are using swift… ;) | 17:40 |
| notmyname | cschwede: s/biased/right/ (FTFY) | 17:40 |
| notmyname | http://whathaveyoutried.com/ and http://greg.brim.net/post/2014/03/01/0118.html are good things to read :-) | 17:44 |
| *** shri1 has joined #openstack-swift | 17:49 | |
| *** shri has quit IRC | 17:51 | |
| *** aix has quit IRC | 17:53 | |
| *** tgohad has joined #openstack-swift | 17:58 | |
| *** lnxnut has quit IRC | 18:00 | |
| *** mattoliverau has quit IRC | 18:00 | |
| *** mattoliverau has joined #openstack-swift | 18:01 | |
| *** tsg has quit IRC | 18:01 | |
| *** elambert has quit IRC | 18:05 | |
| *** geaaru has quit IRC | 18:09 | |
| *** anjuT has joined #openstack-swift | 18:12 | |
| *** lpabon has quit IRC | 18:12 | |
| swift_fan | I've tried reading up on this, but wasn't able to find any leads, anywhere | 18:15 |
| swift_fan | Why is it that Ceph, for instance, supports all 3 ? (object storage, block storage, and file storage). | 18:15 |
| swift_fan | But not Swift ? | 18:15 |
| swift_fan | Is there anything preventing Swift from supporting block storage and file storage, in addition to object storage ? | 18:16 |
| swift_fan | (Ceph is another cluster file system out there). | 18:17 |
| notmyname | swift_fan: for the same reason I can buy http://www.swissarmy.com/us/content/swissarmy/category/1 and http://bernalcutlery.lightspeedwebstore.com/yoshikane-sld-210-gyuto-damascus/dp/1106 | 18:26 |
| *** elambert has joined #openstack-swift | 18:27 | |
| swift_fan | notmyname : So what you're saying is that it's not because Swift can't support all 3 (object storage, block storage, and file storage), but because the developers *choose* not to ? | 18:27 |
| swift_fan | notmyname : (and if so, then what's the reason) | 18:28 |
| ctennis | swift_fan: sometimes the reasons things don't exist is because nobody's had a need to build it | 18:30 |
| ctennis | the swift developers are focused on object store, it's really that simple. there's a limited amount of people working on it, and plenty of work to be done to make it better at what it already does | 18:31 |
| notmyname | ctennis: that. and by focusing on object store we can make sure that is very very good instead of being "distracted" by different use cases like block and file | 18:32 |
| clayg | Sanchit: did some one answer the acl thing for you? | 18:35 |
| *** NM is now known as INFO | 18:42 | |
| *** elambert has quit IRC | 18:46 | |
| *** INFO is now known as NM | 18:47 | |
| *** elambert has joined #openstack-swift | 19:00 | |
| *** foexle has joined #openstack-swift | 19:03 | |
| *** annegent_ has joined #openstack-swift | 19:20 | |
| *** anjuT has quit IRC | 19:20 | |
| swift_fan | notmyname ctennis : Are there any plans in the future to have Swift be able to do cloud *block* storage and cloud *file* storage, as well ? | 19:25 |
| *** annegent_ has quit IRC | 19:27 | |
| *** foexle has quit IRC | 19:27 | |
| glange | no | 19:28 |
| notmyname | http://d.not.mn/monthly_active_swift_contribs-Oct-09-2014.png | 19:35 |
| ctennis | swift_fan: none at this time | 19:37 |
| *** jamespage_ has joined #openstack-swift | 19:37 | |
| *** jamespage_ has quit IRC | 19:38 | |
| peluse | notmyname, cool data | 19:40 |
| swifterdarrell | swift_fan: you might enjoy discussing block storage in #cepth on irc.oftc.net; For real-time chat, the Ceph community gathers in the #ceph channel of the Open and Free Technology Community (OFTC) IRC network. - See more at: http://ceph.com/resources/mailing-list-irc/#sthash.ulGxqZK0.dpuf | 19:40 |
| swift_fan | glange notmyname ctennis : I see that it's probably not desirable in the near future (as you guys have said -- right now there's more than enough tasks to try to make what already exists, even better) -- but isn't it desirable to try to support more things in the longer term future, when the current problems that within the short term have been taken care of / solved / managed effectively ? | 19:40 |
| swifterdarrell | swift_fan: sorry, #ceph | 19:40 |
| glange | it's not realistic | 19:40 |
| glange | different things are different | 19:40 |
| swift_fan | glange : But it was realistic for #ceph, no ? | 19:41 |
| tdasilva | notmyname: what happened 720-800 days ago that caused that big jump in average? | 19:41 |
| swift_fan | for ceph* | 19:41 |
| tdasilva | notmyname: not sure if you can single out something | 19:41 |
| swift_fan | swifterdarrell : Thanks, I'll go check them out as well. | 19:41 |
| swifterdarrell | swift_fan: sweet! | 19:41 |
| swifterdarrell | swift_fan: they would be the experts on that portion of realism | 19:42 |
| glange | swift_fan: it's not a shortcoming of swift that it is what it is -- everything is what it is and is limited by its nature | 19:42 |
| glange | a chair is not a rocket ship, etc | 19:43 |
| swifterdarrell | swift_fan: glange: I sure like notmyname's knife analogy | 19:43 |
| swifterdarrell | swift_fan: glange: ...but who wouldn't want a char... that's ALSO A ROCKETSHIP?! | 19:43 |
| swifterdarrell | *chair | 19:43 |
| glange | haha | 19:44 |
| swifterdarrell | (obviously no one would want a char that was a rocketship) | 19:44 |
| swift_fan | glange : but a chair can be a study chair, rocking chair, and an expandable chair, all at the same time, and be good at all 3, no ? | 19:44 |
| swift_fan | a* | 19:44 |
| glange | sure, but a chair is still limited by its nature in what it can do | 19:44 |
| glange | you just happened to choose some things that it can do | 19:45 |
| swift_fan | glange : ok. But I guess the question is -- forget about all the troubles that may be required to do this -- are there any noticeable advantages for Swift, if Swift were to support cloud *block* storage and cloud *file* storage, as well ? | 19:45 |
| cschwede | swift_fan: there would be disadvantages then. | 19:46 |
| swift_fan | glange : (like the Ceph cluster/network file system). | 19:46 |
| glange | yeah, something that can do more is better -- but you can't make change a thing into any other thing -- there are limits | 19:47 |
| swift_fan | cschwede : I see that there would probably be disadvantages, but given that we agree on that, are there any noticeable advantages that come to mind ? | 19:47 |
| swift_fan | cschwede : positive things. | 19:47 |
| glange | if something is impossible/unrealistic, how long do you think about it before you move on? | 19:47 |
| cschwede | swift_fan: swift solves one problem (and IMO is very good in solving that problem), other storages solve other problems. there is not one solution that fits all | 19:48 |
| swift_fan | glange : I can move on if you want me to ; I was just skeptical about it being "impossible/unrealistic", given that there is a real example out there that does it (for instance, the Ceph cluster/network file system). | 19:49 |
| peluse | "It's kind of fun to do the impossible" -Walt Disney | 19:49 |
| glange | swift_fan: cpeh has its limits to, it can't do some things that swift can | 19:49 |
| cschwede | swift_fan: ceph is strongly consistent, swift eventually. both have advantages and disadvantages and solve different problems | 19:49 |
| ctennis | ceph is a strongly consistent design up front. it started as a block store. they put an object store on top. it doesn't do a very good job of it, in my opinion. | 19:49 |
| glange | and if you made swift into ceph, the new swift wouldn't be able to do some things that the old swift can | 19:49 |
| swift_fan | For one, I imagine that supporting object,block,file storage (all 3 together), would have a good chance of attracting a wider user audience ; that's something that's desirable for the growth of Swift, no ? | 19:50 |
| glange | I guess it would be great if humans made one single object that could do everything that all other objects can do, but that isn't very realistic | 19:50 |
| peluse | ctennis, I'm no ceph expert but I think its the other way around, ceph is an obj store at its foundation | 19:50 |
| cschwede | if you try to solve everything in one solution you get to this „Jack of all trades, master of none“ thing | 19:51 |
| glange | swift_fan: what if by doing that you made it do all 3 poorly because of real life limitations on what is possible? | 19:51 |
| ctennis | peluse: yes I said it backwards | 19:51 |
| swift_fan | glange : That would sort of be like a worst-case scenario, but what's more likely to happen is an average-case scenario for some of them, instead of all 3 doing poorly. | 19:53 |
| swift_fan | glange : And maybe some of it may be able to do very well, still. | 19:53 |
| swift_fan | glange : Even if only 1 type of storage system were supported, even that could do poorly. | 19:54 |
| swift_fan | in the end | 19:54 |
| glange | what if the developers of swift told you you can't do all 3 well? :) | 19:56 |
| peluse | swift_fan, I'm a big believer in "its just SW" and with the right people focused (both time and energy) block and maybe even file abtractions could be added. I think what's missing is all of those ingredients - no real pull and no real interest by anyone that can make it happen | 19:56 |
| notmyname | can we go back to talking about chairs? | 19:58 |
| peluse | ..and the value and or feasbility can be discussed til everyone is blue in the face but I'm thinking w/o a real proposal and the things I mention above its kind of a "moo point" (if anyone missed that episode of Friends its pretty funny) | 19:58 |
| glange | haha | 19:58 |
| ctennis | i like chairs | 19:58 |
| swifterdarrell | swift_fan: you should check out that #ceph channel | 19:59 |
| * peluse just realized he's on vacation.... TTYL | 19:59 | |
| * NM has just updated from 1.13.0 to 2.1.0. Yay! | 19:59 | |
| notmyname | I had an aeron chair once. it was nice, but I like the one in my home office from idea just fine too | 20:00 |
| notmyname | NM: yay! | 20:00 |
| notmyname | s/idea/ikea/ | 20:00 |
| notmyname | ppai: best commit message ever: "Fix unit tests failing in some cases". so in other cases it's not fixed? ;-) | 20:02 |
| ppai | notmyname, ha ha | 20:02 |
| swift_fan | glange : Is that 100% fact that you can't do all 3 well, or is it just a hunch/educated assumptions ? | 20:05 |
| notmyname | swift_fan: glange has a very educated perspective on this. he's been working with it in prod at large scale for years | 20:07 |
| notmyname | swift_fan: the point is that each of these use cases have different fundamental requirements that are in conflict with one another. eg object storage like swift is designed for availability, and that is in conflict with having consistency for a system that supports POSIX | 20:07 |
| *** fifieldt has quit IRC | 20:08 | |
| notmyname | swift_fan: therefore in designing one system for everything, you make tradeoffs instead of doing everything well | 20:08 |
| notmyname | basically, scale demands specialization | 20:08 |
| notmyname | swift_fan: you've kept asking and asking, and monopolizing a lot of time. but, frankly, I don't see that you've demonstrated going and searching for answers or applying answers you've been given. | 20:11 |
| notmyname | swift_fan: we are happy to help you out, as you've seen, but please spend some time researching on your own. there is a lot of stuff out there that is pretty easy to find | 20:11 |
| *** shaifali_ has joined #openstack-swift | 20:11 | |
| *** tkatarki has joined #openstack-swift | 20:11 | |
| tkatarki | portante, ping quick q on swift that maybe u can help or anyone really | 20:12 |
| *** shaifali_ has quit IRC | 20:12 | |
| notmyname | tkatarki: no need to ask to ask. just ask :-) | 20:12 |
| tkatarki | hey john - hi and good to hear from u - will ask | 20:12 |
| tkatarki | so my percona fellow engineer is working on backup tool to backup MySQL to OpenStack Swift. | 20:13 |
| notmyname | tkatarki: nice. I'd love to see that public :-) | 20:14 |
| tkatarki | it is going to be | 20:14 |
| tkatarki | in fact it might be on launchpad already I can find details | 20:14 |
| tkatarki | file size = 522M, reports 20sec for upload file as single object 4 sec for download | 20:15 |
| tkatarki | splits file into chunk size of 10M (52 chunks) and uploads them in three parallel threads. And pushes manifest for dynamic large obj | 20:15 |
| notmyname | tkatarki: may be that the drives or object servers are busy. either with other requests or maybe the background processes | 20:16 |
| tkatarki | says it takes 18 sec to upload all parts however it took 40 seconds to download the entire file in a single thread by accessing the large obj | 20:16 |
| tkatarki | he did some reserahc and found this: https://ask.openstack.org/en/question/2880 | 20:16 |
| tkatarki | "for dynamic large objects there's a default artificial | 20:17 |
| tkatarki | rate_limit_segments_per_second which is turned up kinda high - like 1 | 20:17 |
| tkatarki | req/s after the first 10 segments." | 20:17 |
| *** jamespage_ has joined #openstack-swift | 20:17 | |
| *** jamespage_ has quit IRC | 20:17 | |
| tkatarki | so his question is if he should be using static objects or can he use dynamic large objects and the "slow down" for downlaod from 4 sec to 40 sec is to be expected or is he doing something wrong | 20:18 |
| portante | tkatarki: here | 20:18 |
| tkatarki | hey portante | 20:19 |
| swift_fan | notmyname : I couldn't find much out there in regards to my question, but thanks for your patience. | 20:19 |
| portante | so what is the question, and how are things going? | 20:19 |
| swift_fan | notmyname : :) | 20:20 |
| notmyname | tkatarki: yeah, there's that limit in DLOs. since you're creating a set of objects that don't change over time, then I'd actually recommend the static manifests. in almost every case they are what you should use instead of DLOs | 20:20 |
| tkatarki | ok - that answers the question then; | 20:20 |
| tkatarki | thanks notmyname | 20:20 |
| tkatarki | also, i will ask for the github on this and post it here | 20:20 |
| notmyname | tkatarki: cool, thanks | 20:21 |
| tkatarki | maybe u the siwft community may want to comment | 20:21 |
| *** fifieldt has joined #openstack-swift | 20:21 | |
| notmyname | tkatarki: actually, I'd love to promote that sort of thing as widely as possible. a simple "here's how to back up MySQL into Swift" is a _great_ tool | 20:21 |
| tkatarki | notmyname - that would be super | 20:21 |
| tkatarki | let me get back to you the details | 20:22 |
| tkatarki | we are hoping to have it up for downloads by summit - lets see | 20:22 |
| tkatarki | engineering working on this works from Bangkok - so he is not online now, but, I will try to intro him to this chat tomorrow | 20:23 |
| tkatarki | thanks | 20:23 |
| *** cds has joined #openstack-swift | 20:36 | |
| shri1 | hey guys… is there a way to get stats about how many gets/puts swift received, how many succeeded, etc. | 20:42 |
| notmyname | shri1: you can look at either the logs or the statsd metrics | 20:42 |
| shri1 | for swift to push to statsd, don't I have to enable something in the conf? | 20:43 |
| shri1 | swift-recon? | 20:44 |
| notmyname | shri1: https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L64-L69 | 20:44 |
| notmyname | shri1: note that those settings are in every server config file (proxy, account, container, object) | 20:44 |
| shri1 | awesome… thanks a ton notmyname! | 20:45 |
| *** nshaikh has joined #openstack-swift | 20:45 | |
| *** zul has quit IRC | 21:14 | |
| *** HenryG has quit IRC | 21:14 | |
| *** elambert has quit IRC | 21:17 | |
| *** zul has joined #openstack-swift | 21:19 | |
| *** mahatic__ has quit IRC | 21:20 | |
| mattoliverau | morning all | 21:23 |
| notmyname | mattoliverau: hi | 21:23 |
| *** ppai has quit IRC | 21:24 | |
| notmyname | mattoliverau: I'm just about to go abandon the patches listed | 21:29 |
| mattoliverau | notmyname: cool, well the time stamp on the HTML report is 5 minutes old so it's definitely correct :) | 21:36 |
| *** elambert has joined #openstack-swift | 21:36 | |
| *** ppai has joined #openstack-swift | 21:38 | |
| *** nshaikh has quit IRC | 22:03 | |
| *** annegent_ has joined #openstack-swift | 22:07 | |
| *** annegent_ has quit IRC | 22:07 | |
| *** CaioBrentano has quit IRC | 22:11 | |
| swift_fan | Just wondering, but what are the main object-storage alternatives out there to Swift ? | 22:27 |
| swift_fan | key word being "main", that experts in the field would likely point out. | 22:28 |
| swift_fan | Not just a list of all the object storage systems that exist out there. | 22:28 |
| notmyname | swift_fan: S3 is the biggest | 22:29 |
| swift_fan | notmyname : well, of course S3 | 22:31 |
| *** NM has quit IRC | 22:31 | |
| swift_fan | notmyname : hmm ..... | 22:31 |
| swift_fan | notmyname : I guess this might be a "fuzzy question", but when does regular object-storage become archive-tier storage ? | 22:33 |
| swift_fan | and vice-versa. | 22:33 |
| notmyname | swift_fan: right at the beginning of the deployment, normally. that is, you'd buy and deploy a set of hardware that is good for the use case at the price point you're targeting. | 22:34 |
| notmyname | swift_fan: one of the advantages of swift's storage policy feature is that you can add a different class of storage to your existing cluster | 22:34 |
| openstackgerrit | Prashanth Pai proposed a change to openstack/swift: fsync() on directories https://review.openstack.org/126923 | 22:35 |
| swift_fan | notmyname : hmm..... by "when", I meant characteristics-wise | 22:35 |
| swift_fan | notmyname : as in, when someone points to a storage cluster | 22:35 |
| notmyname | swift_fan: that's what I mean to. so the question right back at you is "what do you consider 'archive'?". figure that out, and you'll know when your cluster meets those goals | 22:36 |
| swift_fan | notmyname : How do they determine, "Hey, that's just regular object storage!", or "Hey, that's archive-tier storage." | 22:36 |
| swift_fan | notmyname : Well, I know that Archive offerings tend to be a lot cheaper, and probably slower access rates, but is there really that much of a difference? | 22:37 |
| swift_fan | notmyname : I mean, technically, regular object-storage is a form of "archiving", too, right ? | 22:38 |
| notmyname | swift_fan: did you have a chance to read that hitachi paper yet? | 22:40 |
| swift_fan | notmyname : Yes, I read it yesterday. | 22:41 |
| swift_fan | notmyname : My impression of it was that it was very general/high-level | 22:41 |
| swift_fan | notmyname : A lot of it seemed to contain very all-swooping visions, such as the parts in section "Rise of Intelligent Storage". | 22:42 |
| swift_fan | notmyname : It probably left me with more questions in the end. | 22:42 |
| swift_fan | notmyname : than I began with. | 22:42 |
| openstackgerrit | Prashanth Pai proposed a change to openstack/swift: fsync() on directories https://review.openstack.org/126923 | 22:43 |
| swift_fan | notmyname : But it was helpful nonetheless! :) | 22:43 |
| notmyname | swift_fan: it does provide a good history of why there are object stores, what problems they are designed to solve (including archiving), and the challenges in creating and running one | 22:44 |
| swift_fan | notmyname : well, in your opinion, is the term "Archive" more a marketing term, to differentiate one type of object-store offering, from another ? | 22:46 |
| notmyname | maybe. but it is a commonly-enough used term for stuff that you need to have but don't frequently access. | 22:48 |
| swift_fan | notmyname : (for instance, Amazon's S3 offering, vs. Amazon's Archive Tier storage offering). | 22:48 |
| notmyname | swift_fan: think of it in terms of application use case. what problem are you trying to solve and what are your constraints? | 22:49 |
| *** Nadeem has quit IRC | 22:49 | |
| swift_fan | notmyname : What are the most common "Archive" oriented types of data off the top of your head ? | 22:50 |
| swift_fan | notmyname : (e.g., system logs?) | 22:51 |
| *** cds has quit IRC | 22:51 | |
| notmyname | swift_fan: "types of data" doesn't mean anything. it's just a bunch of bytes. what matters is the use case. how big is it? how many of it? what's the access patterns? etc | 22:51 |
| notmyname | swift_fan: it's similar to asking "what types of data is this hard drive good for?" | 22:52 |
| swift_fan | notmyname : I see, so once you know and understand the access patterns, for instance, then it's not so important to the storage designer what the actual application is, no ? | 22:53 |
| notmyname | swift_fan: right | 22:53 |
| notmyname | swift_fan: and similarly "when does this hard drive become 'archive'?" doesn't make much sense. it's about the use case and the (often business) constraints | 22:54 |
| *** dmsimard is now known as dmsimard_away | 22:56 | |
| swift_fan | notmyname : Ok, thanks! | 22:58 |
| swift_fan | notmyname : well, by business constraints, you usually mean costs ? | 22:59 |
| swift_fan | notmyname : As in, $ ? | 23:00 |
| *** tdasilva has quit IRC | 23:00 | |
| *** ppai has quit IRC | 23:01 | |
| notmyname | swift_fan: yeah, normally that's the first one. but also legal requirements. and business policies. once you get a storage system deployed into a big company, the actual storage is one of the smaller parts of the system and generally one of the simpler ones ;-) | 23:01 |
| *** HenryG has joined #openstack-swift | 23:07 | |
| swift_fan | notmyname : Thanks. Are there any kinds of data that companies tend to keep around -- forever -- and not just for a long time ? | 23:08 |
| notmyname | yes | 23:08 |
| swift_fan | notmyname : Any most-common-few that you recall ? | 23:08 |
| swift_fan | notmyname : I mean, what data is so important, that it is kept around forever ? | 23:09 |
| notmyname | swift_fan: wouldn't that depend on the company? what data is most important to you that you'd like to keep around forever? | 23:09 |
| swift_fan | notmyname : I see why some data may be kept around for a really really long time, but -- to need to keep something forever ? | 23:09 |
| swift_fan | notmyname : (note, the distinction between 'really really long time', v.s. forever). | 23:10 |
| *** echevemaster has joined #openstack-swift | 23:10 | |
| notmyname | cool. the blue angels just flew over the office | 23:11 |
| *** zackmdavis has left #openstack-swift | 23:11 | |
| *** Murali_ has quit IRC | 23:11 | |
| notmyname | swift_fan: if the distinction between really long time and forever is important, then there are some really interesting papers you can read. but those are outside the scope of swift | 23:13 |
| swift_fan | notmyname : My first guess, for any company, would be financial bookkeeping databases, but do organizations delete those, after a determined lifespan ? | 23:14 |
| swift_fan | notmyname : financial bookkeeping data* | 23:14 |
| notmyname | maybe | 23:16 |
| *** zackmdavis has joined #openstack-swift | 23:20 | |
| ctennis | swift_fan: look at the customer testimonials at https://www.youtube.com/user/SwiftStackVideos, plenty of people espousing their use cases | 23:21 |
| swift_fan | Lastly -- what does kinds of functions does Swift use memcached for, besides managing the tempauth token using memcached ? | 23:21 |
| swift_fan | ctennis -- thank you. | 23:22 |
| swift_fan | I've always been curious about this each time I install and configure memcached. | 23:23 |
| notmyname | swift_fan: https://github.com/openstack/swift/search?utf8=✓&q=cache_from_env | 23:25 |
| *** exploreshaifali has quit IRC | 23:26 | |
| swift_fan | notmyname : Hmm.... this then brings up the question, why does Swift only use memcached for these services, and why not try to use it for everything ? | 23:28 |
| swift_fan | notmyname : because some other service(s) not listed there, may need memcached, moreso than these. | 23:29 |
| swift_fan | notmyname : by services, I mean functions/functionalities. | 23:29 |
| notmyname | http://giphy.com/gifs/zjQrmdlR9ZCM | 23:30 |
| notmyname | swift_fan: if you've got some places in the code that you think could take advantage of memcache, I'd be happy to review a patch | 23:32 |
| swifterdarrell | swift_fan: did you get a chance to chat up #ceph? Maybe they use memcached? | 23:33 |
| swift_fan | notmyname : I mean, nothing special about memcached, right ? It's just a particular form of caching (nothing new). | 23:33 |
| swift_fan | notmyname : So, for instance, why not store objects using memcached ? | 23:33 |
| acorwin | swift_fan: you are aware memcached is not designed to be a persistent storage engine? | 23:34 |
| swift_fan | swifterdarrell : I'll try to do that, and will report back if I get a useful response :) | 23:34 |
| acorwin | swift_fan: and one of the major points of Swift is its exceptional durability & availability. | 23:34 |
| swift_fan | acorwin notmyname : yes, I know that -- obviously, the objects still need to be placed on disk. But why not cache some frequently-read objects using memcached ? | 23:34 |
| *** elambert has quit IRC | 23:35 | |
| acorwin | swift_fan: Wouldn't that merely exacerbate one of your major concerns, about the "eventual" in eventual consistency? | 23:35 |
| swift_fan | acorwin : Not sure, if I understood the question ..... | 23:35 |
| acorwin | swift_fan: well, if you upload an object and I stick it in memcache, that's one more place where I can get out of date, and thus inconsistent, data from | 23:36 |
| ctennis | swift_fan: did you take high school chemistry? | 23:36 |
| swift_fan | acorwin : yeah, but Swift is advertised as "eventual consistency", anyway, so why should that have that much of a negative impact ? | 23:36 |
| acorwin | swift_fan: well that's just one of many reasons it would be a bad idea; i just thought that might be the easiest one to see | 23:37 |
| acorwin | swift_fan: memcached has many other limitations that would make it a very poor choice for caching objects. | 23:37 |
| swift_fan | acorwin : And maybe, one of the first places to update the object would be in the memcached ? | 23:37 |
| acorwin | swift_fan: Keeping all the locations of an object up to date and consistent is a very hard problem. Adding more would make it even harder. | 23:38 |
| swift_fan | acorwin : That's the thing -- I don't know what it is about memcached -- which seems like it's just another caching method at its fundamentals -- that makes it not a great option for storing object data ..... | 23:39 |
| ctennis | swift_fan: the reason I ask is because, alongside the class learning and question/answer there is also lab and experimentation, which is important. it feels like your "why not" questions are not going to be answered until you dive in directly and spend the time to understand, without the discussion. | 23:39 |
| swift_fan | acorwin : That was in reference to the previous message you wrote, | 23:39 |
| swift_fan | acorwin : . | 23:39 |
| swift_fan | ctennis : I see. | 23:40 |
| acorwin | swift_fan: memcache is a cache in memory (thus the "mem") | 23:40 |
| acorwin | swift_fan: you can't just stick as much data as you want in it with no repercussions. | 23:40 |
| swift_fan | acorwin : Yes, that's clear. But it's essentially a cache system, in essence. | 23:40 |
| acorwin | swift_fan: So? | 23:40 |
| swift_fan | acorwin : Right, so you would use it only for the very most-read/accessed objects. | 23:41 |
| swift_fan | acorwin : However much you can get away with, right ? | 23:41 |
| acorwin | swift_fan: what if the most read objects are large? what if memcache starts expunging objects as fast as you put new ones in, causing nothign but giant memory thrashing? what if you have a write-heavy workload where memcache is basically never valuable? | 23:41 |
| swifterdarrell | swift_fan: maybe Ceph uses memcache for objects? | 23:41 |
| acorwin | swift_fan: or, perhaps, to turn this on its head: what would Swift *gain* by trying to stick memcache everywhere? I think i've pointed out plenty of negatives at this point. | 23:41 |
| zackmdavis | ctennis, the importance of empiricism was one of the things I learned in high school chemistry too. There was even a slogan about it: Resist the Temptation to Forget to Measure | 23:44 |
| acorwin | zackmdavis: that's a mouthful, I wonder if it could be conveniently shortened? | 23:44 |
| ctennis | Skip the Labs and you Fail the Class? | 23:46 |
| notmyname | ctennis: I think rtfm is more catchy that slfc | 23:46 |
| ctennis | says you | 23:47 |
| *** nellysmitt has quit IRC | 23:47 | |
| *** kyles_ne has quit IRC | 23:49 | |
| *** kyles_ne has joined #openstack-swift | 23:50 | |
| swift_fan | acorwin : I see from your explanation that using memcached for storing objects can lead to a lot of potential problems. But is there no option/feature at all, in Swift, that you can turn on which will allow you to store some objects in memcached ? | 23:50 |
| swifterdarrell | swift_fan: why would you do that to yourself? | 23:50 |
| acorwin | swift_fan: no, there is not. I would guess that the overlap between use cases where memcached is valuable and the use cases where swift is valuable is near zero | 23:51 |
| swift_fan | acorwin : And if there isn't, are there any plans to enable in the future that option/feature in Swift ? | 23:51 |
| acorwin | swift_fan: I can't answer that with any authority, but no, there is not. | 23:51 |
| swifterdarrell | swift_fan: honestly, I'd me curious if anyone in #memcached could speak to plans for moving memcached into the Object Storage space | 23:51 |
| swifterdarrell | swift_fan: http://memcached.org/ says, "If you are curious about something, feel free to ask on the IRC channel - #memcached on freenode." | 23:53 |
| swift_fan | acorwin : So, can you have a fully-functional, good-performing Swift cluster, even if you don't install/configure memcached on your Swift cluster nodes ? | 23:53 |
| notmyname | that's not what he said | 23:53 |
| acorwin | swift_fan: No, because swift installs and configures that on its own, for the purposes it needs memcached for. | 23:53 |
| swifterdarrell | swift_fan: I don't think *anyone* in #memcached would agree with that | 23:54 |
| swifterdarrell | swift_fan: but it could be worth checking | 23:54 |
| *** kyles_ne has quit IRC | 23:55 | |
| *** dmsimard_away is now known as dmsimard | 23:55 | |
| swift_fan | Ok, thank you guys :) | 23:56 |
| ctennis | have a nice evening | 23:57 |
| notmyname | swift_fan: what time zone are you in? | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
