| notmyname | piousbox: I'd love to. and if you're in the SF area, I'd be happy to get coffee and talk face to face | 00:02 |
|---|---|---|
| piousbox | notmyname: I'm in San Jose | 00:02 |
| piousbox | notmyname: where are you? ;) | 00:02 |
| zaitcev | SF of course. All the cool kids moved there. San Jose is not cool anymore. | 00:03 |
| notmyname | heh | 00:03 |
| notmyname | I gotta run. I'll be back later this evening. (riding home on my bike like the cool kids in SF do) | 00:03 |
| zaitcev | what, no Google bus for you? | 00:04 |
| piousbox | Cool cool. Bicycles are very important :) | 00:17 |
| piousbox | notmyname: I'm downtown SF this weekend. We should exchange emails 'n stuff. | 00:17 |
| piousbox | notmyname: piousbox at gmail is me. | 00:17 |
| piousbox | zaitcev: I like SJ | 00:18 |
| piousbox | zaitcev: I actually moved out of SF a few years ago. | 00:18 |
| *** mmcardle has joined #openstack-swift | 00:27 | |
| *** mmcardle has quit IRC | 00:31 | |
| *** zaitcev has quit IRC | 00:39 | |
| *** shri has left #openstack-swift | 00:39 | |
| notmyname | piousbox: ah. I'll be out of town this weekend on a mini vacation with my family. maybe later | 00:46 |
| notmyname | ah! topic update time! | 00:46 |
| *** ChanServ changes topic to "Current Swift Release: 1.13.1 | Priority Reviews: https://wiki.openstack.org/wiki/Swift/PriorityReviews | Channel Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/" | 00:47 | |
| *** chuck_ has quit IRC | 00:55 | |
| *** Edward-Zhang has joined #openstack-swift | 01:26 | |
| *** mmcardle has joined #openstack-swift | 01:29 | |
| *** mmcardle has quit IRC | 01:33 | |
| *** nosnos has joined #openstack-swift | 01:33 | |
| *** piousbox has quit IRC | 01:34 | |
| *** bill_az has quit IRC | 01:35 | |
| *** Edward-Zhang has quit IRC | 01:35 | |
| *** simpleAJ has joined #openstack-swift | 01:41 | |
| *** simpleAJ2 has joined #openstack-swift | 01:43 | |
| simpleAJ2 | Hi | 01:43 |
| simpleAJ2 | does anybody know if I would be able to install swift on single debian vm or may be two debian vm to start with using this guide | 01:44 |
| simpleAJ2 | http://docs.openstack.org/developer/swift/howto_installmultinode.html | 01:44 |
| simpleAJ2 | I just want to get my hands dirty and play with it | 01:44 |
| creiht | simpleAJ2: if you wanted to just install on one node to try out, I recommend http://docs.openstack.org/developer/swift/development_saio.html | 01:45 |
| notmyname | simpleAJ2: https://github.com/swiftstack/vagrant-swift-all-in-one is a good way to get started quickly too | 01:46 |
| simpleAJ2 | it need to be strictly one node..I can bring two three nodes for sure..but just want to install as easily as possible ;) | 01:46 |
| notmyname | creiht: you'd probably like that too, actually | 01:46 |
| simpleAJ2 | oh good ..will start with that then | 01:46 |
| creiht | notmyname: yeah I've been meaning to check it out | 01:46 |
| notmyname | simpleAJ2: ya, both of those are one VM with simulated 4 servers in it | 01:46 |
| simpleAJ2 | ok..great..otherwise..with very simple setup...would I be needing atleast 5 nodes.. | 01:47 |
| simpleAJ2 | with replication factor say just 1.. | 01:47 |
| creiht | simpleAJ2: really depends on your use case, and how much redundancy, and performance you need | 01:47 |
| simpleAJ2 | replication would 1..this just prototyping phase | 01:48 |
| creiht | then you could easily set that up on one machine | 01:48 |
| simpleAJ2 | but that should give me some idea how to go about it | 01:48 |
| simpleAJ2 | ok sounds good | 01:48 |
| simpleAJ2 | and one more question that I am meaning to ask is that..(sorry if I haven't gone through documentation that well enough).. | 01:49 |
| simpleAJ2 | question is | 01:49 |
| simpleAJ2 | in proxy server..will I be able to intercept the client request and decide which storage node to be used like out of 10..these 5 are good for you client | 01:50 |
| creiht | simpleAJ2: can you expand what your use case is? | 01:50 |
| simpleAJ2 | sure.. | 01:50 |
| creiht | do you want like fast(ssd) and slow(sata) storage, or something like that? | 01:50 |
| simpleAJ2 | no..actually that would be something like those nodes would be closer to client..have less congested bandwidth and may be less loaded servers | 01:51 |
| creiht | well I think you could use the region support to do that | 01:51 |
| creiht | but I'm not as familiar with those bits | 01:52 |
| creiht | notmyname could probably help you more in that area | 01:52 |
| simpleAJ2 | ok very well then. thank you for your help..will digg into region servers to see if that can help me | 01:52 |
| *** mwstorer has quit IRC | 01:54 | |
| simpleAJ2 | notmyname: in case you have any pointers let me know..It seems that I would need to override some module that makes this decision..because things like network congestion or experiment data affinity would like to be available from external sources..and I would need to somehow find a way to tell that to proxy server | 01:56 |
| simpleAJ2 | s/would like/would likely | 01:57 |
| *** changbl has joined #openstack-swift | 02:22 | |
| *** zhiyan_ is now known as zhiyan | 02:24 | |
| *** mmcardle has joined #openstack-swift | 02:29 | |
| notmyname | simpleAJ: I'm not sure I follow your use case yet | 02:32 |
| simpleAJ | oh | 02:32 |
| *** simpleAJ2 has quit IRC | 02:33 | |
| simpleAJ | basically when client would like to write an object | 02:33 |
| simpleAJ | if I got it correctly proxy server determines which of the storage nodes it would like to write to. | 02:33 |
| simpleAJ | may be that decision made based on regions and zones. | 02:34 |
| *** mmcardle has quit IRC | 02:35 | |
| simpleAJ | if that is correct, then I would like to see if we can have pluggable decision making module..like that will tell proxy server .. these are the nodes that you would want to write to for certain reasons | 02:36 |
| glange | where an object "lives" in swift is too fundamental to how it works to put the object in some sort of arbitary location, I think | 02:37 |
| simpleAJ | ok | 02:37 |
| glange | swift calculates a number based on the object name/path and then looks up in the ring which servers the object lives on | 02:37 |
| glange | and that is used by replication, when the object is read, etc | 02:38 |
| simpleAJ | that is during read right? | 02:38 |
| simpleAJ | i want to control where to write too. | 02:38 |
| glange | it used during writes, reads, heads, posts, replication, etc | 02:38 |
| glange | by everything | 02:38 |
| simpleAJ | interesting didn't know that | 02:38 |
| portante | simpleAJ, what is coming is storage polices, which will let you construct different rings for differing kinds of offerings | 02:39 |
| simpleAJ | ok sounds reasonable | 02:40 |
| portante | so you might have a ring of SSDs, another ring of 7200rpm, 10rpm, etc., and then you associated a policy with a container | 02:40 |
| portante | so you might have gold, silver, bronze, for SSD, 10krpm and 7krpm, and you associate the "gold" with container A, charging customers more for that then the bronze level | 02:41 |
| portante | or something like that scheme | 02:41 |
| portante | storage policies will enable stuff like Erasure Codes, or other storage backends that come available | 02:41 |
| simpleAJ | ok | 02:42 |
| simpleAJ | just a thought that..these things look like static properties.. | 02:42 |
| simpleAJ | basically what I am looking at is dynamic nature of the environment. where network is not that stable | 02:42 |
| glange | simpleAJ: I think what you want is not in swift now and might never be in swift | 02:42 |
| simpleAJ | hmm :) | 02:43 |
| glange | but it's good to want things :) | 02:43 |
| simpleAJ | yeah | 02:43 |
| simpleAJ | I think these pointers are good to get started with swift..thanks a lot people | 02:44 |
| notmyname | simpleAJ: for an overview of some concepts, check out the videos and blog posts and especially the architecture page at https://swiftstack.com/openstack-swift/ | 02:47 |
| simpleAJ | awesome that looks helpful | 02:47 |
| notmyname | simpleAJ: what you're asking for is sort of like what's being built for storage policies (but I think that's a little too complicated for what you're looking for), and sorta like the existing "affinity" feature | 02:48 |
| simpleAJ | yeah right. | 02:48 |
| simpleAJ | basically i am grad student..and exploring what are effects of really unstable network on storage systems.. | 02:49 |
| simpleAJ | and I found swift kinda of familiar with my concepts..so I am trying to explore that :) | 02:49 |
| glange | http://tinyurl.com/o5ue649 | 02:49 |
| glange | that is what the ring looks like | 02:49 |
| glange | the object ring | 02:49 |
| notmyname | simpleAJ: realize that the purpose of swift is to abstract away storage volumes so you have a single namespace, scalable storage system, where you, as the consumer (client), don't have to worry about pools of storage, filesystem limits, or hardware failures | 02:49 |
| simpleAJ | :D | 02:49 |
| simpleAJ | right agree | 02:50 |
| notmyname | durable storage, built for scale, and high availability and massive aggregate concurrency. basically, if you have user-generated content served on the web, you need swift | 02:51 |
| simpleAJ | ok | 02:53 |
| *** matsuhas_ has joined #openstack-swift | 02:53 | |
| *** matsuhashi has quit IRC | 02:53 | |
| *** matsuhas_ has quit IRC | 02:54 | |
| *** matsuhashi has joined #openstack-swift | 02:56 | |
| *** matsuhashi has quit IRC | 03:02 | |
| *** matsuhashi has joined #openstack-swift | 03:04 | |
| *** scoorp has joined #openstack-swift | 03:11 | |
| *** simpleAJ has left #openstack-swift | 03:13 | |
| notmyname | simpleAJ: found it. if you're interested specifically in network issues, here's a couple of old videos of talks I've given on swift failure handing. https://www.youtube.com/watch?v=Z7z_1gBsoAc and http://mirror.linux.org.au/linux.conf.au/2013/mp4/Playing_with_OpenStack_Swift.mp4 | 03:13 |
| *** nosnos has quit IRC | 03:17 | |
| *** cheri has joined #openstack-swift | 03:26 | |
| *** zackf has joined #openstack-swift | 03:31 | |
| *** mmcardle has joined #openstack-swift | 03:31 | |
| *** mmcardle has quit IRC | 03:35 | |
| openstackgerrit | A change was merged to openstack/swift: Fixed error in container sync exc handling https://review.openstack.org/85197 | 03:53 |
| *** matsuhashi has quit IRC | 04:01 | |
| *** scoorp has quit IRC | 04:02 | |
| *** cheri has quit IRC | 04:03 | |
| *** cheri has joined #openstack-swift | 04:04 | |
| *** matsuhashi has joined #openstack-swift | 04:05 | |
| *** zackf has quit IRC | 04:14 | |
| *** chandan_kumar has joined #openstack-swift | 04:16 | |
| *** europack has joined #openstack-swift | 04:17 | |
| *** cheri has quit IRC | 04:19 | |
| *** nosnos has joined #openstack-swift | 04:20 | |
| openstackgerrit | Jenkins proposed a change to openstack/swift: Updated from global requirements https://review.openstack.org/81207 | 04:28 |
| *** h6w has joined #openstack-swift | 04:28 | |
| h6w | Hallo ppl. | 04:29 |
| h6w | I had to change the IPs of a couple of storage nodes. | 04:29 |
| h6w | So I called set_info on the IP and replication IP. All good according to swift-ring-builder. | 04:29 |
| h6w | However, if I do swift-recon, it seems to use the old IP addresses. | 04:29 |
| h6w | Where is that stored? | 04:30 |
| *** mmcardle has joined #openstack-swift | 04:33 | |
| *** cheri has joined #openstack-swift | 04:33 | |
| *** mmcardle has quit IRC | 04:38 | |
| *** madhuri has joined #openstack-swift | 04:40 | |
| *** chandan_kumar has quit IRC | 05:06 | |
| *** ben00b has joined #openstack-swift | 05:06 | |
| *** ben00b has quit IRC | 05:08 | |
| *** chandan_kumar has joined #openstack-swift | 05:08 | |
| *** nshaikh has joined #openstack-swift | 05:24 | |
| *** zhiyan is now known as zhiyan_ | 05:25 | |
| *** mmcardle has joined #openstack-swift | 05:34 | |
| *** mmcardle has quit IRC | 05:38 | |
| *** Guest__ has joined #openstack-swift | 05:58 | |
| *** Guest__ is now known as bach | 06:04 | |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Merge container storage_policy_index values. https://review.openstack.org/83942 | 06:12 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Keep status_changed_at up-to-date with status changes. https://review.openstack.org/85186 | 06:12 |
| *** bach has quit IRC | 06:13 | |
| *** Guest__ has joined #openstack-swift | 06:18 | |
| *** madhuri has quit IRC | 06:23 | |
| *** mmcardle has joined #openstack-swift | 06:29 | |
| *** madhuri has joined #openstack-swift | 06:31 | |
| *** mmcardle has quit IRC | 06:31 | |
| *** mmcardle has joined #openstack-swift | 06:31 | |
| *** mmcardle has quit IRC | 06:35 | |
| *** Guest__ has quit IRC | 06:38 | |
| *** bvandenh has joined #openstack-swift | 06:38 | |
| Anju | clayg : ping | 06:40 |
| *** madhuri has quit IRC | 06:41 | |
| *** cheri has quit IRC | 06:41 | |
| *** ppai has joined #openstack-swift | 06:52 | |
| *** ashish_ has joined #openstack-swift | 06:56 | |
| *** europack has quit IRC | 06:57 | |
| *** chandankumar_ has joined #openstack-swift | 06:59 | |
| ashish_ | Hey EveryOne I got the foolowing error when running the "ssbench-worker -h" command in the ssbench benchmarking tool. paste.openstack.org/show/75059/ | 07:06 |
| ashish_ | Please help. | 07:06 |
| *** matsuhashi has quit IRC | 07:07 | |
| *** saju_m has joined #openstack-swift | 07:12 | |
| *** mmcardle has joined #openstack-swift | 07:13 | |
| *** Longgeek_ has joined #openstack-swift | 07:21 | |
| hugokuo | ashish_: you need newer greenlet version | 07:23 |
| *** chandankumar_ has quit IRC | 07:23 | |
| hugokuo | ashish_: pip freeze | grep green ; sudo pip install --upgrade greenlet | 07:27 |
| hugokuo | hope it help | 07:27 |
| *** cheri has joined #openstack-swift | 07:32 | |
| *** nacim has joined #openstack-swift | 07:40 | |
| *** zhiyan_ is now known as zhiyan | 07:56 | |
| *** ashish_ has quit IRC | 07:58 | |
| *** Longgeek_ has quit IRC | 07:59 | |
| *** als has joined #openstack-swift | 07:59 | |
| als | hi I have a problem debugging proxy-server | 08:00 |
| als | i am doing remote deubgging with pydev | 08:00 |
| als | i can see the log in the debugger saying that 'the conection is attached' | 08:00 |
| als | but when the client makes a request | 08:00 |
| als | to the proxy server | 08:01 |
| als | the proxy cant respond | 08:01 |
| als | i put my code in | 08:01 |
| als | swift / swift / common / wsgi.py | 08:02 |
| als | i put there pydev.settrace ... | 08:02 |
| als | and that connects back to the debugger | 08:02 |
| als | my question is, how do you set up proxy-server debugging? | 08:03 |
| *** matsuhashi has joined #openstack-swift | 08:04 | |
| *** Midnightmyth has joined #openstack-swift | 08:06 | |
| *** tanee-away is now known as tanee | 08:12 | |
| *** Longgeek_ has joined #openstack-swift | 08:14 | |
| *** als has left #openstack-swift | 08:14 | |
| openstackgerrit | A change was merged to openstack/swift: Read the configuration once for all func tests https://review.openstack.org/84332 | 08:28 |
| *** Longgeek_ has quit IRC | 08:31 | |
| *** cheri has quit IRC | 08:35 | |
| *** matsuhashi has quit IRC | 08:35 | |
| *** fbo_away is now known as fbo | 08:40 | |
| *** tanee is now known as tanee-away | 08:41 | |
| *** nacim has quit IRC | 08:42 | |
| *** cheri has joined #openstack-swift | 08:51 | |
| *** matsuhashi has joined #openstack-swift | 08:53 | |
| *** nacim has joined #openstack-swift | 08:54 | |
| openstackgerrit | Yuan Zhou proposed a change to openstack/swift: Fixes versioning function tests with non-zero default policy https://review.openstack.org/82515 | 08:54 |
| openstackgerrit | Victor Stinner proposed a change to openstack/python-swiftclient: Python 3: fix tests on HTTP headers https://review.openstack.org/84711 | 08:57 |
| *** saurabh_ has quit IRC | 09:08 | |
| *** jamie_h has joined #openstack-swift | 09:18 | |
| *** mkollaro has joined #openstack-swift | 09:21 | |
| *** nacim has quit IRC | 09:42 | |
| *** matsuhashi has quit IRC | 09:47 | |
| *** matsuhashi has joined #openstack-swift | 09:47 | |
| *** Longgeek_ has joined #openstack-swift | 09:50 | |
| *** nacim has joined #openstack-swift | 09:56 | |
| *** saju_m has quit IRC | 09:56 | |
| *** cheri has quit IRC | 09:58 | |
| *** Longgeek_ has quit IRC | 09:59 | |
| *** Longgeek_ has joined #openstack-swift | 10:00 | |
| *** ppai has quit IRC | 10:07 | |
| *** Longgeek_ has quit IRC | 10:08 | |
| *** saju_m has joined #openstack-swift | 10:09 | |
| *** saju_m has quit IRC | 10:14 | |
| *** matsuhashi has quit IRC | 10:23 | |
| *** nacim has quit IRC | 10:28 | |
| *** nacim has joined #openstack-swift | 10:29 | |
| *** saju_m has joined #openstack-swift | 10:30 | |
| *** matsuhashi has joined #openstack-swift | 10:30 | |
| *** mkollaro has quit IRC | 10:31 | |
| *** nosnos has quit IRC | 10:33 | |
| *** Trixboxer has joined #openstack-swift | 10:34 | |
| *** saju_m has quit IRC | 10:35 | |
| *** saju_m has joined #openstack-swift | 10:35 | |
| *** matsuhashi has quit IRC | 10:38 | |
| *** chandankumar_ has joined #openstack-swift | 10:39 | |
| *** nosnos has joined #openstack-swift | 10:39 | |
| *** saju_m has quit IRC | 10:40 | |
| *** saju_m has joined #openstack-swift | 10:40 | |
| *** matsuhas_ has joined #openstack-swift | 10:41 | |
| *** cheri has joined #openstack-swift | 10:43 | |
| *** matsuhas_ has quit IRC | 10:52 | |
| *** matsuhashi has joined #openstack-swift | 10:52 | |
| *** tanee-away is now known as tanee | 10:54 | |
| *** chandankumar_ has quit IRC | 10:55 | |
| *** matsuhashi has quit IRC | 10:57 | |
| *** chandankumar_ has joined #openstack-swift | 11:12 | |
| *** cheri has quit IRC | 11:27 | |
| *** nosnos has quit IRC | 11:30 | |
| openstackgerrit | Peter Portante proposed a change to openstack/swift: Move module level setup under setup_package method https://review.openstack.org/84339 | 11:31 |
| openstackgerrit | Peter Portante proposed a change to openstack/swift: In-process swift server for functional tests https://review.openstack.org/66108 | 11:31 |
| *** saju_m has quit IRC | 11:38 | |
| openstackgerrit | Jenkins proposed a change to openstack/swift: Updated from global requirements https://review.openstack.org/81207 | 11:57 |
| *** mmcardle has quit IRC | 11:58 | |
| *** mkollaro has joined #openstack-swift | 12:04 | |
| *** saju_m has joined #openstack-swift | 12:10 | |
| *** saju_m has quit IRC | 12:14 | |
| *** saju_m has joined #openstack-swift | 12:15 | |
| *** mmcardle1 has joined #openstack-swift | 12:18 | |
| *** JuanManuelOlle has joined #openstack-swift | 12:18 | |
| *** tdasilva has left #openstack-swift | 12:24 | |
| *** tanee is now known as tanee-away | 12:30 | |
| *** JuanManuelOlle1 has joined #openstack-swift | 12:32 | |
| *** mmcardle1 has quit IRC | 12:35 | |
| *** JuanManuelOlle has quit IRC | 12:35 | |
| *** mmcardle has joined #openstack-swift | 12:43 | |
| *** mmcardle1 has joined #openstack-swift | 12:44 | |
| *** mmcardle has quit IRC | 12:45 | |
| *** chandankumar_ has quit IRC | 12:47 | |
| *** Fin1te has joined #openstack-swift | 12:49 | |
| *** saju_m has quit IRC | 12:57 | |
| *** Fin1te has quit IRC | 12:57 | |
| openstackgerrit | Florent Flament proposed a change to openstack/python-swiftclient: Update help message to specify unit of --segment-size option. https://review.openstack.org/85335 | 13:07 |
| openstackgerrit | Florent Flament proposed a change to openstack/python-swiftclient: Update help message to specify unit of --segment-size option. https://review.openstack.org/85335 | 13:11 |
| *** JuanManuelOlle1 has quit IRC | 13:18 | |
| *** PradeepChandani has quit IRC | 13:18 | |
| *** krtaylor has joined #openstack-swift | 13:24 | |
| *** lpabon has joined #openstack-swift | 13:36 | |
| *** piyush1 has joined #openstack-swift | 13:40 | |
| *** piyush2 has joined #openstack-swift | 13:44 | |
| *** tdasilva has joined #openstack-swift | 13:45 | |
| *** piyush1 has quit IRC | 13:46 | |
| openstackgerrit | Madhuri Kumari proposed a change to openstack/swift: Bypass committing pending records from .pending file https://review.openstack.org/85351 | 13:48 |
| *** madhuri has joined #openstack-swift | 13:51 | |
| madhuri | Can anyone please help reviewing this https://review.openstack.org/#/c/84819/ ? | 13:51 |
| openstackgerrit | Donagh McCabe proposed a change to openstack/swift: Reclaim containers even if account db is reclaimed https://review.openstack.org/84696 | 13:53 |
| cschwede | madhuri: hmm, maybe I miss something here - but https://review.openstack.org/#/c/84819/ is adding some methods that aren’t used? | 14:02 |
| *** foexle has joined #openstack-swift | 14:12 | |
| *** Longgeek_ has joined #openstack-swift | 14:17 | |
| *** zackf has joined #openstack-swift | 14:30 | |
| *** mwstorer has joined #openstack-swift | 14:30 | |
| *** saju_m has joined #openstack-swift | 14:31 | |
| *** a_hacker2 has joined #openstack-swift | 14:33 | |
| *** saju_m has quit IRC | 14:34 | |
| a_hacker2 | creiht: I think I mave have found it. http://paste.openstack.org/show/lyAaCCUArlEUvvypWba8/ | 14:34 |
| *** saju_m has joined #openstack-swift | 14:35 | |
| a_hacker2 | creiht: the md5's of the ring files are the same on all boxes | 14:39 |
| *** ashish_ has joined #openstack-swift | 14:42 | |
| *** mkerrin has quit IRC | 14:42 | |
| ashish_ | hugokuo thanks | 14:42 |
| creiht | a_hacker2: well either one of your services is using the wrong ring, or you have another cluster or something misconfigured so you have cross-talk causing weird issues | 14:56 |
| creiht | something weird is going on | 14:56 |
| a_hacker2 | yes - i'm pretty confused | 14:57 |
| creiht | I've heard that you guys have had issues before where internal ips get re-used or something like that so that there is cross-talk between different swift clusters | 14:57 |
| creiht | or something like that | 14:58 |
| creiht | don't remember the exact specifics | 14:58 |
| a_hacker2 | yeah, this is a new environment in a seperate datacenter :( | 14:58 |
| creiht | but either way, either one of the services are getting the wrong ring somehow, or something external is interfering | 15:00 |
| a_hacker2 | i can't find/think of anything external. i also made a new ring with only 1 machine and just 3 disks. | 15:00 |
| a_hacker2 | i think it's the account service/replicator that is somehow getting the wrong ring data | 15:01 |
| *** joeljwright has joined #openstack-swift | 15:01 | |
| creiht | the other possibility is that some of the servers have the wrong hash prefix/suffix on some of the servers | 15:01 |
| a_hacker2 | checking that | 15:02 |
| creiht | a_hacker2: also when you isolated everything to one box, did you remove that box from the original cluster? (to make sure the original cluster isn't trying to make requests to that box) | 15:08 |
| creiht | In other news, I'm really disliking Requests right now :/ | 15:10 |
| openstackgerrit | Florent Flament proposed a change to openstack/python-swiftclient: Update help message to specify unit of --segment-size option. https://review.openstack.org/85335 | 15:11 |
| *** IRTermite has quit IRC | 15:13 | |
| *** IRTermite has joined #openstack-swift | 15:14 | |
| *** IRTermite has quit IRC | 15:23 | |
| *** lpabon has quit IRC | 15:26 | |
| *** nshaikh has quit IRC | 15:32 | |
| *** tanee-away is now known as tanee | 15:34 | |
| *** saju_m has quit IRC | 15:35 | |
| *** Finite has joined #openstack-swift | 15:35 | |
| a_hacker2 | creiht: looks like that was it. swift.conf is different on the machines | 15:38 |
| a_hacker2 | TYVM! | 15:38 |
| creiht | there you go | 15:39 |
| creiht | np | 15:39 |
| *** JuanManuelOlle has joined #openstack-swift | 15:40 | |
| *** swiftishard has joined #openstack-swift | 15:40 | |
| *** Finite has left #openstack-swift | 15:43 | |
| *** Finite has joined #openstack-swift | 15:43 | |
| a_hacker2 | Finite! | 15:44 |
| *** dmartls1 has joined #openstack-swift | 15:44 | |
| Finite | a_hacker2 1 | 15:45 |
| Finite | !!! | 15:45 |
| openstack | Finite: Error: "!!" is not a valid command. | 15:45 |
| gholt | creiht: Once again, swift.conf on the attack. | 15:54 |
| a_hacker2 | creiht: sorry for wasting your time. I'm writing a config test for this condition right now. | 15:55 |
| gholt | a_hacker2: That has happened to many, so don't feel bad at all. | 15:55 |
| gholt | creiht's got some ideas to try to make it more obvious when that is the issue. | 15:56 |
| *** RockKuo_TW has quit IRC | 16:01 | |
| *** russellb is now known as rustlebee | 16:06 | |
| *** Finite_ has joined #openstack-swift | 16:08 | |
| *** JuanManuelOlle has quit IRC | 16:14 | |
| *** JuanManuelOlle has joined #openstack-swift | 16:17 | |
| dmartls1 | Running swift-proxy-1.12 with apache/wsgi, uploading unsegmented objects works fine, however when trying to segment the object is complains about the segment size being too large and reports the size of the entire object (before segmenting), is there some special config I'm missing to support DLO and wsgi? | 16:24 |
| creiht | a_hacker2: yeah it is an easy problem to have | 16:25 |
| *** csd_ has joined #openstack-swift | 16:29 | |
| *** bill_az has joined #openstack-swift | 16:33 | |
| joeljwright | Hi everyone, as I mentioned a few weeks ago, I've been working on refactoring the python-swiftclient into an importable library, and updating the swift binary (now shell.py) to use that library. The updated code is available here: https://github.com/sohonetlabs/python-swiftclient/tree/swift-service and I'm hoping this could be integrated into the python-swiftclient project. | 16:33 |
| joeljwright | Obviously the patch is huge, so there will be work to so yet | 16:33 |
| joeljwright | wanted to get feedback on the patch and advice on submitting the update | 16:34 |
| joeljwright | The code in that branch is based on the latest code in openstack/python-swiftclient master | 16:36 |
| *** Longgeek_ has quit IRC | 16:36 | |
| creiht | joeljwright: can you give a short example of how it is different? | 16:38 |
| zackmdavis | joeljwright, um, I thought python-swiftclient already was an importable library; "from swiftclient import client" | 16:38 |
| creiht | for example you could already use swiftclient as a library | 16:39 |
| creiht | yeah what zackmdavis said | 16:39 |
| joeljwright | well, there was also a lot of logic in the swift binary | 16:39 |
| creiht | a *lot* of projects already use it that way | 16:39 |
| joeljwright | I haven't touched the code in client.py at all | 16:39 |
| joeljwright | all I've done is create a new SwiftService class that incorporates all the logic from the swift binary | 16:40 |
| *** mwstorer has quit IRC | 16:41 | |
| creiht | ahh | 16:41 |
| joeljwright | and updated the multithreading code to allow multiple operatons | 16:41 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Fix issue with account broker and pre-SPI databases https://review.openstack.org/83840 | 16:42 |
| *** chandankumar_ has joined #openstack-swift | 16:42 | |
| *** mwstorer has joined #openstack-swift | 16:42 | |
| joeljwright | so any code that uses client.py is fine, but there would be an extra layer to allow new projects to have access to multi-threaded upload/download from a simple import | 16:42 |
| joeljwright | sorry, that should have been upload/download etc. | 16:43 |
| joeljwright | the porting of the swift binary to the library was a proof of concept | 16:44 |
| creiht | hrm | 16:46 |
| joeljwright | I'm hoping it will also help with testing the functionality from shell.py | 16:46 |
| *** Guest__ has joined #openstack-swift | 16:48 | |
| creiht | so I'm paging through the original shell.py code and *most* of the code is about parsing command line options, and mapping that to client.py calls | 16:48 |
| creiht | sorry if I'm missing something (which is entirely likely since my allergies are killing me today) | 16:48 |
| creiht | can you help me understand what added functionality one would getting using service.py over client.py? | 16:49 |
| joeljwright | well, for example, in shell.py st_upload is ~300 lines of logic not included in client.py | 16:50 |
| joeljwright | using service.py you only have to specify container and an object list to get that logic and a multithreaded upload | 16:51 |
| creiht | ahh I see | 16:51 |
| creiht | ok thanks | 16:51 |
| creiht | sorry I'm a bit slow today :/ | 16:51 |
| joeljwright | :) | 16:51 |
| joeljwright | np | 16:51 |
| joeljwright | I ended up creating this because I was making calls out to the swift binary from within scripts | 16:52 |
| creiht | so why not refactor most of that code into client.py rather than having another wrapping class? | 16:52 |
| joeljwright | but I wanted better control over threading | 16:52 |
| creiht | understood | 16:52 |
| joeljwright | ultimately it started as a separate project, but I thought it might be useful here | 16:52 |
| joeljwright | and I didn't want to mess with client.py because so many people are using it as is | 16:53 |
| creiht | yeah I can see the use of pulling the extra logic out of shell.py | 16:53 |
| *** Guest__ has quit IRC | 16:54 | |
| joeljwright | I'm not expecting this to be an easy process | 16:54 |
| joeljwright | just wanted to start the discussion now that I have something reasonable to read and based off an up to date shell.py | 16:55 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Fix issue with account broker and pre-SPI databases https://review.openstack.org/83840 | 16:56 |
| joeljwright | the patch itself is huge, and now doubt introduces new bugs, but it has been very useful to me | 16:56 |
| creiht | joeljwright: I'll have to take a look when my brain is a bit less cloudy | 16:57 |
| creiht | :) | 16:57 |
| joeljwright | np | 16:57 |
| creiht | but the only question left in my mind right now, is if it is better to refactor those things into client.py, or introduce a wrapping as you did in service.py | 16:57 |
| joeljwright | creiht: so that leads me onto another question: what's the process for continuing this discussion? | 16:58 |
| creiht | well I think the general idea is a reasonable one | 16:58 |
| creiht | the next step is proposing a patch through gerrit | 16:58 |
| creiht | https://wiki.openstack.org/wiki/Gerrit_Workflow | 16:59 |
| *** csd_ has quit IRC | 16:59 | |
| creiht | That describes the overall process | 16:59 |
| joeljwright | ok, thanks very much | 16:59 |
| creiht | once a patch is proposed, I imagine you will get a lot more feedback | 16:59 |
| joeljwright | I'll read though that - I already have the required accounts I think, it's just a case of working out how to use them | 16:59 |
| creiht | hehe | 17:00 |
| creiht | yeah the process is a bit daunting at first | 17:00 |
| creiht | if you run into issues, feel free to ask in here and we can help | 17:00 |
| joeljwright | Thanks | 17:01 |
| *** tanee is now known as tanee-away | 17:01 | |
| *** csd_ has joined #openstack-swift | 17:01 | |
| *** mmcardle1 has quit IRC | 17:05 | |
| *** chandankumar_ has quit IRC | 17:09 | |
| *** shri has joined #openstack-swift | 17:11 | |
| *** nacim has quit IRC | 17:13 | |
| *** mmcardle has joined #openstack-swift | 17:21 | |
| *** mmcardle has quit IRC | 17:26 | |
| clayg | joeljwright: I looked at that change when you mentioned it before and I'm sure it only has gotten better - I'm quite ready to review on gerrit | 17:32 |
| clayg | joeljwright: as you said it's a big patch, so it may take some time to get it all merged in - sooner we get started the better! | 17:33 |
| *** mmcardle has joined #openstack-swift | 17:38 | |
| *** mmcardle has quit IRC | 17:42 | |
| *** zhiyan is now known as zhiyan_ | 17:42 | |
| *** ashish_ has quit IRC | 17:43 | |
| joeljwright | clayg: thanks for the enthusiasm, I've started working through the gerrit process, but this is the first time I've done it so it might take me a while | 17:46 |
| *** mmcardle has joined #openstack-swift | 17:52 | |
| peluse | clayg: time to discuss the | 17:52 |
| peluse | fix spi container thing? | 17:52 |
| *** joeljwright has left #openstack-swift | 17:53 | |
| *** mmcardle has quit IRC | 17:57 | |
| *** openstackgerrit has quit IRC | 18:01 | |
| *** openstackgerrit has joined #openstack-swift | 18:02 | |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Fix issue with account broker and pre-SPI databases https://review.openstack.org/83840 | 18:03 |
| creiht | so it looks like the way we are using requests in python-swiftclient, it is generating a new connection for each request | 18:06 |
| creiht | I think we need to figure out how to use the session stuf in the client | 18:06 |
| *** Gue______ has joined #openstack-swift | 18:07 | |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Merge container storage_policy_index values. https://review.openstack.org/83942 | 18:07 |
| creiht | or maybe not.... | 18:07 |
| creiht | heh | 18:07 |
| creiht | ok yes we are | 18:09 |
| creiht | hehe | 18:09 |
| creiht | I think the HTTPConnection class needs to use a requests session instad of calling requests.request directly to keep the connection open | 18:11 |
| Alex_Gaynor | creiht: I think there's a WIP review for doing that | 18:14 |
| creiht | Alex_Gaynor: oh really? link? | 18:14 |
| Alex_Gaynor | creiht: https://review.openstack.org/#/c/74444/ | 18:14 |
| creiht | Alex_Gaynor: hah yeah | 18:15 |
| creiht | that's more or less what I was thinking of doing, but I think they put the session in the wrong place | 18:16 |
| *** a_hacker2 has quit IRC | 18:16 | |
| creiht | or maybe that is | 18:18 |
| creiht | stupid allergies | 18:18 |
| creiht | Alex_Gaynor: thanks | 18:18 |
| creiht | that does seem to improve things a bit | 18:21 |
| clayg | peluse: yeah just got out of a meeting - do you think we should open up another card to fix migrations and just take out the update for now? | 18:22 |
| peluse | one sec | 18:23 |
| clayg | peluse: it turns out that spi on the un-migrated rows just get's left as null - not sure if we could do something in the alter to make it fill in 0's | 18:23 |
| peluse | clayg: so the update was working for me - meaning if I have an old DB that didn't have the col and I then run with the patch, the update affects all rows that had nothing in them and puts a 0 in them | 18:25 |
| peluse | clayg: the only strange thing was that I also added a test (likes your I think but I didn't look) and with the test code the assert we have as a SWL thing didn't hit (catching the attempt to update). It only hit for me when I tested it "for real" | 18:26 |
| clayg | peluse: a) I'm pretty sure there was no test case covering that b) that doesn't seem to work for me, and I believe that makes sense seeing as how the containers table explicitly forbids any update's | 18:27 |
| peluse | SWL=SQL... new keyboard and I can't seem to use it | 18:27 |
| clayg | lol | 18:27 |
| clayg | peluse: in my test I had to insert the row by hand before the migration since put_container would always trigger the migration when inserting the row - i.e. it takes some manual sql to make a pre-spi-schema database that has some rows in it | 18:28 |
| peluse | clayg: why did you make this comment "we can't just alter the table and stuff in default values with an update," - it does work or do you mean we shouldn't be doing that for some reason | 18:28 |
| peluse | clayg: ya, that's what I did too. | 18:29 |
| clayg | the test I wrote shows it doesn't work, and that makes sense to me since the containers table forbids updates | 18:29 |
| peluse | clayg: :) Yes, I was going to ask why it forbids it, clearly I thought it was OK to 'ignore that' for our case since we do have a valid reason to do an UPDATE (I tought/think) | 18:30 |
| * peluse is going to buy a new keyboard at lunchtime | 18:31 | |
| clayg | oh... but do you disable it before the migration? | 18:31 |
| peluse | clayg: not sure what you mean, wanna hop on the phone for a few? | 18:32 |
| clayg | peluse: skype - yeah one sec | 18:34 |
| peluse | believe it or not I don't use skype | 18:34 |
| clayg | peluse: heh | 18:34 |
| peluse | we have Msft Lync at work :( | 18:35 |
| peluse | 480-554-3688 if you can call from skype | 18:36 |
| *** Gue______ has quit IRC | 18:36 | |
| peluse | if not I think the answer to your question is yes , in _migrate_add_storage_policy_index I simply ignore the error that the 'verbotten code' is preventing when we do the UPDATE | 18:36 |
| peluse | hmm, maybe I can use lync to call sykpe after all... checking | 18:37 |
| clayg | wow | 18:38 |
| *** Gue______ has joined #openstack-swift | 18:40 | |
| *** byeager has joined #openstack-swift | 18:41 | |
| *** jamie_h has quit IRC | 18:45 | |
| anticw | peluse: @work they luvs the lync too :) i used to uninstall on the windows lappy but they keep reinstalling it when i turn my back | 18:47 |
| *** csd_ has quit IRC | 18:48 | |
| *** mmcardle has joined #openstack-swift | 18:53 | |
| *** mkollaro has quit IRC | 18:53 | |
| *** csd_ has joined #openstack-swift | 18:54 | |
| *** mmcardle has quit IRC | 18:57 | |
| openstackgerrit | Joel Wright proposed a change to openstack/python-swiftclient: Add importable SwiftService incorporating shell.py logic https://review.openstack.org/85453 | 19:16 |
| *** Gue______ has quit IRC | 19:22 | |
| *** joeljwright has joined #openstack-swift | 19:26 | |
| *** joeljwright has left #openstack-swift | 19:27 | |
| openstackgerrit | Joel Wright proposed a change to openstack/python-swiftclient: Add importable SwiftService incorporating shell.py logic https://review.openstack.org/85453 | 19:29 |
| openstackgerrit | Peter Portante proposed a change to openstack/swift: Attempt to ensure connect always timesout https://review.openstack.org/85457 | 19:30 |
| *** IRTermite has joined #openstack-swift | 19:39 | |
| *** fbo is now known as fbo_away | 19:41 | |
| openstackgerrit | Joel Wright proposed a change to openstack/python-swiftclient: Add importable SwiftService incorporating shell.py logic https://review.openstack.org/85453 | 19:42 |
| *** Gue______ has joined #openstack-swift | 19:48 | |
| peluse | anticw: yeah, I can't get away from Windows at work. Forced to run it on my Mac if you can believe it (actually does quite well) | 19:48 |
| *** mmcardle has joined #openstack-swift | 19:54 | |
| *** piyush2 has quit IRC | 19:57 | |
| *** mmcardle has quit IRC | 19:59 | |
| *** Trixboxer has quit IRC | 20:03 | |
| *** jamie_h has joined #openstack-swift | 20:07 | |
| *** piyush has joined #openstack-swift | 20:08 | |
| *** piyush1 has joined #openstack-swift | 20:13 | |
| *** piyush has quit IRC | 20:14 | |
| *** Gue______ has quit IRC | 20:17 | |
| *** csd_ has quit IRC | 20:26 | |
| portante | zaitcev: you around? | 20:28 |
| portante | glange: you around? | 20:32 |
| portante | Finally got https://review.openstack.org/84339 through the check jobs, can you approve? | 20:32 |
| *** csd_ has joined #openstack-swift | 20:32 | |
| *** Guest___ has joined #openstack-swift | 20:34 | |
| *** Guest___ has quit IRC | 20:38 | |
| *** mmcardle has joined #openstack-swift | 20:55 | |
| *** serveascode has joined #openstack-swift | 20:59 | |
| *** mmcardle has quit IRC | 20:59 | |
| *** serveascode has quit IRC | 21:00 | |
| *** JuanManuelOlle has quit IRC | 21:11 | |
| *** tdasilva has left #openstack-swift | 21:16 | |
| peluse | clayg: you there? | 21:18 |
| clayg | peluse: sure sure | 21:19 |
| *** piyush1 has left #openstack-swift | 21:20 | |
| peluse | so your test code behaves for me as it does for you... | 21:21 |
| peluse | the reason why my initial attempt to add test code like yours didn't work is (I think) because I was using in mem DB = that's why I went and tested it manually | 21:21 |
| peluse | strange | 21:21 |
| clayg | stupid memdb's - can't trust 'em | 21:21 |
| peluse | I will re-test manually to make sure I wasn't seeing things that I wanted to :) | 21:21 |
| clayg | lol | 21:22 |
| peluse | will let ya know | 21:22 |
| *** zaitcev has joined #openstack-swift | 21:25 | |
| *** ChanServ sets mode: +v zaitcev | 21:25 | |
| *** csd_ has quit IRC | 21:25 | |
| *** byeager has quit IRC | 21:26 | |
| peluse | clayg: I'm not seeing things. The UDPATE really does work when testing "for real". Here are my exact staps (maybe you can spot something fishy) | 21:29 |
| *** fbo_away is now known as fbo | 21:30 | |
| peluse | 1) checkout master (2) resetswift (4) start swift (5) put cont called 'mycont' (6) usw sqlbroser to confirm DB has no SPI and container is in DB container table (7) checkout fix bracch, with update included (8) restart but not reset swift (9) put container called 'mycont2' (10) look at DB and confirm both containers are in table and both have a vluae of 0 for SPI | 21:31 |
| peluse | (not that I put enough times to force merge) | 21:31 |
| peluse | dmanit, nott=note | 21:32 |
| clayg | peluse: hehehehe | 21:32 |
| peluse | ugh | 21:32 |
| clayg | peluse: so that does look pretty sound, maybe you can push your backend changes over the gerrit review and I can pull down and try and duplicate your results? | 21:32 |
| clayg | sqlite --version -> 3.7.9 2011-11-01 00:52:41 c7c6050ef060877ebe77b41d959e9df13f8c9b5e | 21:32 |
| clayg | ^ fwiw | 21:32 |
| peluse | qq: how do I check my sqlite version? :) | 21:33 |
| *** byeager has joined #openstack-swift | 21:33 | |
| clayg | like does maybe your fix branch remove the "disallow update" triggery thing? cause i don't know exactly how that gets applied for realzys | 21:34 |
| *** Finite has quit IRC | 21:34 | |
| *** Finite_ has quit IRC | 21:34 | |
| peluse | nope, its there | 21:34 |
| *** byeager_ has joined #openstack-swift | 21:34 | |
| clayg | peluse: i just said "sqlite --version" and it spit out a value - dunno if that's sufficient, but it does seem like a change in behavior between my functional test and yours | 21:34 |
| clayg | ... so it could be differences in the implementation - or something else - i'm reaching | 21:35 |
| *** serverascode has joined #openstack-swift | 21:35 | |
| peluse | but my test fails like yours does though | 21:35 |
| peluse | meaning, with patch I can see that the row w/the old container does not get updated... however when I rmeove the 'prevent update' deal I still don't see the row updated when the test code checks it. Will verify that again real quick | 21:36 |
| clayg | hrmm.... but seems different functionally so idk - I'd like to try and duplicate your results functionally, but I already verified it womm functionally the way it does in the test (with my backend changes, so i want to retry with yours) | 21:36 |
| peluse | same exact sqlite BTW | 21:36 |
| clayg | peluse: k, well if you wanna push it up on gerrit (or somewhere else I can nab your working tree) I'll try it out functionally on my setup | 21:37 |
| peluse | OK, will do. Will double check a few things first | 21:37 |
| *** byeager has quit IRC | 21:38 | |
| *** mmcardle has joined #openstack-swift | 21:57 | |
| *** csd_ has joined #openstack-swift | 22:00 | |
| *** mmcardle has quit IRC | 22:01 | |
| peluse | clayg: I think I see why manual testing works.... | 22:08 |
| peluse | its not the UPDATE thats getting the spi updated, its in _commit_puts_load | 22:09 |
| clayg | peluse: oh i missed your bit on same sql version, so that rules out that i 'spose | 22:10 |
| clayg | yeah if the .pending file is processed against the new code it would definately get the right value in spi | 22:10 |
| peluse | yup | 22:10 |
| clayg | bah, i need to rebase torgomatic keep-per-object-spi on the latest changes in feature/ec | 22:11 |
| peluse | so at least that solves the 'difference between test and func' behavior and solidifies that we can't blow off the integrity error | 22:11 |
| peluse | so is it possible for the scenario created by the current test code to happen for real? I mean, all DB merges come through the pending file right? | 22:12 |
| peluse | (so all will get spi of 0 if they don't have the col) | 22:13 |
| clayg | peluse: yeah i mean a db that's been around awhile is going to have those rows in the database not in the .pending file | 22:17 |
| clayg | peluse: maybe I don't quite follow what you're asking, or i'm still missing something about your functional test | 22:18 |
| peluse | clayg: damned existing installed base | 22:18 |
| peluse | clayg: you actually just asnwered my question | 22:18 |
| clayg | peluse: bummer | 22:19 |
| peluse | clayg: so I think I have a handle on this if you'd like me to take a stab at getting it right. Have to head to football practice (son, not me) so will be over the weekend sometime | 22:19 |
| clayg | peluse: i'm not so sure there's such a rush - we need to get it squared of course - but ultimately it all needs to work and I need to stay on the reconciler - so have fun! | 22:20 |
| peluse | rock n roll. as usual, thanks for the insights.... | 22:21 |
| *** byeager_ has quit IRC | 22:21 | |
| *** byeager has joined #openstack-swift | 22:22 | |
| *** byeager_ has joined #openstack-swift | 22:23 | |
| *** byeager has quit IRC | 22:23 | |
| *** csd_ is now known as csd | 22:26 | |
| *** byeager_ has quit IRC | 22:27 | |
| *** byeager has joined #openstack-swift | 22:27 | |
| *** byeager has quit IRC | 22:31 | |
| *** techguru has joined #openstack-swift | 22:42 | |
| *** fbo is now known as fbo_away | 22:43 | |
| *** Midnightmyth has quit IRC | 22:45 | |
| *** jamie_h has quit IRC | 22:47 | |
| *** Fin1te has joined #openstack-swift | 22:55 | |
| *** mwstorer has quit IRC | 22:55 | |
| *** mmcardle has joined #openstack-swift | 22:58 | |
| *** mmcardle has quit IRC | 23:02 | |
| *** dmartls1 has quit IRC | 23:10 | |
| *** zackf has quit IRC | 23:18 | |
| *** techguru has quit IRC | 23:20 | |
| *** mmcardle has joined #openstack-swift | 23:51 | |
| *** mmcardle1 has joined #openstack-swift | 23:53 | |
| *** mmcardle has quit IRC | 23:53 | |
| *** Fin1te has quit IRC | 23:55 | |
| *** foexle has quit IRC | 23:57 | |
| *** mmcardle1 has quit IRC | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!