| *** zaitcev__ has joined #openstack-swift | 00:09 | |
| *** ChanServ sets mode: +v zaitcev__ | 00:09 | |
| *** Guest28904 has quit IRC | 00:34 | |
| *** NM has joined #openstack-swift | 00:50 | |
| *** gkadam has joined #openstack-swift | 01:16 | |
| *** baojg has joined #openstack-swift | 01:22 | |
| *** diablo_rojo has quit IRC | 01:23 | |
| *** gkadam has quit IRC | 01:40 | |
| *** ianychoi has joined #openstack-swift | 02:00 | |
| openstackgerrit | Merged openstack/swift master: Set Content-Type with s3api metadata updates. https://review.opendev.org/658945 | 02:55 |
|---|---|---|
| *** NM has quit IRC | 03:01 | |
| *** gkadam has joined #openstack-swift | 03:23 | |
| *** gkadam has quit IRC | 03:23 | |
| *** psachin has joined #openstack-swift | 03:41 | |
| *** blmt has quit IRC | 03:42 | |
| *** ianychoi has quit IRC | 03:49 | |
| *** ianychoi has joined #openstack-swift | 03:49 | |
| *** ianychoi has quit IRC | 03:51 | |
| *** ianychoi has joined #openstack-swift | 03:52 | |
| *** zaitcev__ has quit IRC | 04:39 | |
| *** zaitcev__ has joined #openstack-swift | 04:51 | |
| *** ChanServ sets mode: +v zaitcev__ | 04:51 | |
| *** pcaruana has joined #openstack-swift | 05:20 | |
| *** baojg has quit IRC | 06:01 | |
| *** baojg has joined #openstack-swift | 06:36 | |
| *** hseipp has joined #openstack-swift | 06:47 | |
| *** tesseract has joined #openstack-swift | 07:08 | |
| *** rdejoux has joined #openstack-swift | 07:10 | |
| *** rcernin has quit IRC | 07:19 | |
| *** ccamacho has joined #openstack-swift | 07:27 | |
| openstackgerrit | Alexandre Lécuyer proposed openstack/swift master: Specify pickle protocol in REPLICATE() https://review.opendev.org/659614 | 07:41 |
| *** zaitcev__ has quit IRC | 08:20 | |
| *** kota_ has quit IRC | 08:26 | |
| *** kota_ has joined #openstack-swift | 08:26 | |
| *** ChanServ sets mode: +v kota_ | 08:26 | |
| *** ccamacho has quit IRC | 08:27 | |
| *** zaitcev__ has joined #openstack-swift | 08:31 | |
| *** ChanServ sets mode: +v zaitcev__ | 08:31 | |
| *** tkajinam has quit IRC | 08:44 | |
| *** baojg has quit IRC | 08:45 | |
| *** baojg has joined #openstack-swift | 08:50 | |
| *** ccamacho has joined #openstack-swift | 08:59 | |
| *** baojg has quit IRC | 09:06 | |
| *** baojg has joined #openstack-swift | 09:07 | |
| *** zaitcev_ has joined #openstack-swift | 09:07 | |
| *** ChanServ sets mode: +v zaitcev_ | 09:07 | |
| *** zaitcev__ has quit IRC | 09:11 | |
| *** e0ne has joined #openstack-swift | 09:17 | |
| *** baojg has quit IRC | 09:32 | |
| *** zaitcev__ has joined #openstack-swift | 09:42 | |
| *** ChanServ sets mode: +v zaitcev__ | 09:42 | |
| *** zaitcev_ has quit IRC | 09:46 | |
| *** rcernin has joined #openstack-swift | 09:50 | |
| *** baojg has joined #openstack-swift | 11:02 | |
| *** henriqueof has joined #openstack-swift | 11:06 | |
| *** rcernin has quit IRC | 11:36 | |
| *** zaitcev__ has quit IRC | 11:56 | |
| *** hseipp has quit IRC | 12:00 | |
| *** zaitcev__ has joined #openstack-swift | 12:08 | |
| *** ChanServ sets mode: +v zaitcev__ | 12:08 | |
| *** ccamacho has quit IRC | 12:15 | |
| *** cwright has quit IRC | 12:42 | |
| *** cwright has joined #openstack-swift | 12:43 | |
| *** ccamacho has joined #openstack-swift | 13:25 | |
| *** zaitcev__ is now known as zaitcev | 13:41 | |
| *** NM has joined #openstack-swift | 14:03 | |
| *** baojg has quit IRC | 14:16 | |
| *** baojg has joined #openstack-swift | 14:16 | |
| *** baojg has quit IRC | 14:16 | |
| *** baojg has joined #openstack-swift | 14:17 | |
| *** baojg has quit IRC | 14:17 | |
| *** baojg has joined #openstack-swift | 14:18 | |
| *** baojg has quit IRC | 14:19 | |
| *** baojg has joined #openstack-swift | 14:19 | |
| *** baojg has quit IRC | 14:19 | |
| *** baojg has joined #openstack-swift | 14:21 | |
| *** baojg has quit IRC | 14:21 | |
| *** baojg has joined #openstack-swift | 14:22 | |
| *** baojg has quit IRC | 14:23 | |
| *** baojg has joined #openstack-swift | 14:23 | |
| *** baojg has quit IRC | 14:23 | |
| *** baojg has joined #openstack-swift | 14:24 | |
| *** baojg has quit IRC | 14:24 | |
| *** baojg has joined #openstack-swift | 14:25 | |
| *** baojg has quit IRC | 14:25 | |
| *** baojg has joined #openstack-swift | 14:25 | |
| *** baojg has quit IRC | 14:26 | |
| *** baojg has joined #openstack-swift | 14:26 | |
| *** baojg has quit IRC | 14:27 | |
| *** baojg has joined #openstack-swift | 14:27 | |
| *** baojg has quit IRC | 14:27 | |
| *** baojg has joined #openstack-swift | 14:28 | |
| *** baojg has quit IRC | 14:28 | |
| *** baojg has joined #openstack-swift | 14:28 | |
| *** baojg has quit IRC | 14:29 | |
| *** baojg has joined #openstack-swift | 14:29 | |
| *** baojg has quit IRC | 14:30 | |
| *** baojg has joined #openstack-swift | 14:31 | |
| *** baojg has quit IRC | 14:31 | |
| *** baojg has joined #openstack-swift | 14:32 | |
| *** baojg has quit IRC | 14:32 | |
| *** baojg has joined #openstack-swift | 14:32 | |
| *** baojg has quit IRC | 14:33 | |
| *** baojg has joined #openstack-swift | 14:33 | |
| *** baojg has quit IRC | 14:34 | |
| *** baojg has joined #openstack-swift | 14:34 | |
| *** baojg has quit IRC | 14:34 | |
| *** baojg has joined #openstack-swift | 14:35 | |
| *** baojg has quit IRC | 14:35 | |
| *** baojg has joined #openstack-swift | 14:36 | |
| *** baojg has quit IRC | 14:36 | |
| *** baojg has joined #openstack-swift | 14:36 | |
| *** baojg has quit IRC | 14:37 | |
| *** baojg has joined #openstack-swift | 14:37 | |
| *** baojg has quit IRC | 14:38 | |
| *** baojg has joined #openstack-swift | 14:38 | |
| *** baojg has quit IRC | 14:38 | |
| *** baojg has joined #openstack-swift | 14:39 | |
| *** baojg has quit IRC | 14:39 | |
| *** baojg has joined #openstack-swift | 14:40 | |
| *** baojg has quit IRC | 14:41 | |
| *** baojg has joined #openstack-swift | 14:41 | |
| *** baojg has quit IRC | 14:42 | |
| *** baojg has joined #openstack-swift | 14:42 | |
| *** baojg has quit IRC | 14:42 | |
| *** baojg has joined #openstack-swift | 14:43 | |
| *** baojg has quit IRC | 14:43 | |
| *** gyee has joined #openstack-swift | 15:34 | |
| openstackgerrit | Alexandre Lécuyer proposed openstack/swift feature/losf: Add small unit tests for vfile and related modules https://review.opendev.org/659515 | 15:58 |
| notmyname | good morning | 16:14 |
| notmyname | 🎵Happy birthday to Swift!🎵 | 16:14 |
| alecuyer | hello! happy birthday swift :-) | 16:15 |
| *** tesseract has quit IRC | 16:17 | |
| *** psachin has quit IRC | 16:17 | |
| *** tesseract has joined #openstack-swift | 16:18 | |
| timburke | 🎉 | 16:21 |
| *** tesseract has quit IRC | 16:58 | |
| *** tesseract has joined #openstack-swift | 16:59 | |
| *** tesseract has quit IRC | 17:17 | |
| *** e0ne has quit IRC | 17:17 | |
| openstackgerrit | Merged openstack/swift master: Make build/upload/promote image jobs non-voting https://review.opendev.org/659391 | 17:27 |
| openstackgerrit | Tim Burke proposed openstack/swift master: Be explicit that a code path is just for py2 https://review.opendev.org/659851 | 17:30 |
| tdasilva | timburke: what was the decision about libec release? | 17:43 |
| timburke | tdasilva, we should definitely do that. haven't gotten to it yet though :-) | 17:50 |
| *** Alphazero_ has joined #openstack-swift | 17:51 | |
| tdasilva | timburke: was going to offer to help, i can start push buttons and prepare release notes... | 17:51 |
| Alphazero_ | Hi all - I'm having real trouble installing Swift with JuJu - I keep getting the following error with Swift Proxy but I'm not sure what the problem is: http://paste.openstack.org/show/751530/ | 17:51 |
| timburke | tdasilva, that'd be great! thanks | 17:51 |
| Alphazero_ | This is what my Swift proxy and Swift-Storage config file looks like: http://paste.openstack.org/show/751531/ | 17:51 |
| Alphazero_ | Any help or pointers would be appreciated thanks! | 17:52 |
| timburke | Alphazero_, like notmyname I'm not real familiar with JuJu... am i right in thinking that the storage nodes are waiting on the proxy to come up? but then it looks like the proxy is refusing because it doesn't know about any storage nodes... hmm... | 17:54 |
| Alphazero_ | thats correct - I was initially under the impression I was launching too few swift containers / too few swift zones - but with 15 of them up (3 per zone) - I can't see that being the issue | 17:56 |
| timburke | i wonder what `zone-assignment: auto` means/does... | 17:56 |
| Alphazero_ | I think here: https://jaas.ai/swift-proxy/precise/46 - it automatically allocated new swift units to zones, in accordance with "balancing" | 17:58 |
| Alphazero_ | I originally had it set to manual and was manually allotting units to zones - but that's resulted in the same issue | 17:58 |
| Alphazero_ | this is the last last step of my openstack deployment (3 week long project) so keen to get it resolved :) | 17:59 |
| timburke | huh. well, i definitely like to see more people deploying swift! let's see what we can do... | 18:00 |
| Alphazero_ | I was originally following this guide - https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/install-openstack.html - but swift has been been giving me problems there too | 18:00 |
| timburke | so what are the juju commands you've run to get to this point? | 18:00 |
| Alphazero_ | this is effectively the order of commands: http://paste.openstack.org/show/751532/ | 18:05 |
| timburke | seems to make sense given the docs... i wonder if we can get some insight into what juju thinks the ring assignments ought to look like | 18:11 |
| Alphazero_ | I'm new to that part I'm afraid - I actually started out trying to build it with just one swift node - but I haven't been able to get swift-proxy and swift-storage to talk to each other successfully no matter how many or how few units or zones are deployed | 18:13 |
| Alphazero_ | nothing fails but this message is where it sits "Not enough storage zones for minimum replicas" and then everything becomes idle | 18:15 |
| Alphazero_ | and there's not a whole lot to change in the config file :) | 18:16 |
| timburke | i wonder if it'd work with a minimal sort of a cluster: just one replica and one unit each for proxy and storage | 18:16 |
| timburke | it really feels like a live-lock though: the proxy unit is blocked waiting for storage, but the storage doesn't want to come up until there's a proxy :-/ | 18:18 |
| timburke | interesting... there are separate checks for # of nodes and # of zones: https://github.com/openstack/charm-swift-proxy/blob/cc280d9/lib/swift_utils.py#L1209-L1215 | 18:23 |
| timburke | so it knows about the nodes... but is getting confused about the zones... | 18:23 |
| timburke | looks like there are two failure paths in has_minimum_zones: https://github.com/openstack/charm-swift-proxy/blob/bec2714/swift_manager/manager.py#L183-L199 | 18:25 |
| timburke | so, is there any logging about "Not enough zones (X) defined to satisfy minimum replicas (need >= Y)"? do we know whether we'd see INFO level logs? | 18:26 |
| timburke | if *not*... i guess it means the ring file doesn't exist yet? | 18:27 |
| timburke | or maybe it's looking for a ring builder file... | 18:28 |
| openstackgerrit | Merged openstack/swift master: Wait longer for log lines in unit test https://review.opendev.org/652791 | 18:30 |
| timburke | wow... i wonder when these got lifted: https://github.com/openstack/charm-swift-proxy/blob/bec2714/swift_manager/manager.py#L206-L230 | 18:30 |
| timburke | and at what point (if ever) they'd be able to move to the "new" builder format... | 18:31 |
| Alphazero_ | http://paste.openstack.org/show/751536/ - these are the and debug logs related to Swift-proxy shown by "Juju Debug-log" | 18:33 |
| timburke | or... no... i was thinking of the ring file that has a new format... i've just been meaning to get around to doing a new format for the builder, too :P | 18:34 |
| timburke | huh. so ctxt['allowed_hosts'] knows about the storage nodes... but when we open up the ring builder, it doesn't have any devs at all... | 18:36 |
| Alphazero_ | Could this be it - is there supposed to be relation between the Swift-storage zones - and units added? | 18:37 |
| Alphazero_ | unit-swift-storage-zone2-5: 19:35:13 INFO unit.swift-storage-zone2/5.juju-log proxy relations's interface, swift-storage, is related but has no units in the relation. | 18:37 |
| *** itlinux has joined #openstack-swift | 18:38 | |
| timburke | that'd make some sense. i woulda thought your `sudo juju add-unit --to lxd:3 swift-storage-zone2` would have added a unit for the zone, but ¯\_(ツ)_/¯ | 18:41 |
| Alphazero_ | haha same here | 18:41 |
| Alphazero_ | but I can't seem to individually create those relations | 18:43 |
| Alphazero_ | plus as you rightly say the command seems to do (atleast from how it appears in juju status) what its supposed to do | 18:44 |
| timburke | do you see any logs about "Added new device to ring ..."? | 18:48 |
| Alphazero_ | I originally had it setup as manual - I might take it down and and rerun the whole thing with auto set in the proxy file... | 18:48 |
| Alphazero_ | I'll take a loot | 18:48 |
| Alphazero_ | *look | 18:48 |
| *** mvkr has joined #openstack-swift | 18:49 | |
| timburke | or "Storage relation changed -processing" | 18:49 |
| timburke | or anything about devices "already processed" or "already in the ring" or "already mounted", or "Format device ... failed" | 18:56 |
| Alphazero_ | aha! | 18:56 |
| timburke | i love those moments! | 18:57 |
| Alphazero_ | http://paste.openstack.org/show/751540/ | 18:59 |
| Alphazero_ | I think I've messed up somewhere in defining the block device | 18:59 |
| Alphazero_ | it funny because after going through config changed state - it basically kept waiting the proxy - but I guess all was not well! | 19:00 |
| Alphazero_ | In my swift beginners mind I thought the block device was just the partition of the physical device? | 19:01 |
| Alphazero_ | what MAAS shows is the following: https://pasteboard.co/If9Kuaz.png | 19:03 |
| Alphazero_ | if you know what I should be setting the block device as that would be helpful :) - given that swift-storage units are running in containers I don't fully understand how they would interact with a physical disk - but I guess that was a heroic solution on my end! | 19:06 |
| zaitcev | Swift consumes a filesystem, not a block device. So, containers can bind-mount from the host. | 19:08 |
| timburke | well, the /dev/sda-part1 definitely looks fishy -- i would've expected something more like /dev/sda1, which would imply you'd want just 'block-device: sda1' | 19:08 |
| timburke | but i'm also more than a little fuzzy on how containers are supposed to be interacting with physical disks ;-) | 19:09 |
| timburke | zaitcev, poking at the charms source, it looks like they take care of formatting as xfs | 19:09 |
| zaitcev | They can bind-mount /dev too. Usually it's done so host's /dev/log is visible. But if so, they can also access /dev/sda. It can be hazardous if you manage to start two of them though. | 19:10 |
| timburke | yeah, i was just thinking that | 19:12 |
| Alphazero_ | the juju docs for the swift proxy charm have this as the "block device": "/etc/swift/storage.img|2G" | 19:17 |
| Alphazero_ | but is that something that gets created in the container (on its filesystem)? | 19:17 |
| timburke | i think so -- i'm guessing that's why they say "Good for development and testing." re: that format on https://jaas.ai/swift-storage | 19:18 |
| timburke | presumably, for prod you'd want a real disk | 19:19 |
| Alphazero_ | hmm so that hasn't worked it gives a very clear error with that ie "/etc/swift/storage.img|2G" set as the block device | 19:47 |
| Alphazero_ | https://pastebin.com/bkNai72m | 19:48 |
| Alphazero_ | http://paste.openstack.org/show/751545/ - basically all the swift-storage containers have error'd out | 19:49 |
| Alphazero_ | I'm tempted to go for "guess" now that I've seen that as an option! :P | 19:49 |
| Alphazero_ | within the debug log - this I think is the key line: unit-swift-storage-zone2-7: 20:49:52 DEBUG unit.swift-storage-zone2/7.config-changed subprocess.CalledProcessError: Command '['losetup', '--find', '/etc/swift/storagedev1.img']' returned non-zero exit status 1. unit-swift-storage-zone2-7: 20:49:52 ERROR juju.worker.uniter.operation hook "config-changed" failed: exit status 1 | 19:51 |
| *** pcaruana has quit IRC | 20:02 | |
| *** itlinux has quit IRC | 20:06 | |
| *** zaitcev_ has joined #openstack-swift | 20:11 | |
| *** ChanServ sets mode: +v zaitcev_ | 20:11 | |
| *** zaitcev has quit IRC | 20:15 | |
| notmyname | http://d.not.mn/swift_cake_9th_bday.jpg <--- cake for the birthday! | 20:16 |
| *** zaitcev_ is now known as zaitcev | 20:36 | |
| *** rdejoux has quit IRC | 20:37 | |
| *** Alphazero_ has quit IRC | 20:45 | |
| zaitcev | timburke: so, how about we accept patch 657505 (alt-bulk)? | 20:53 |
| patchbot | https://review.opendev.org/#/c/657505/ - swift - py3: bulk -- alternate approach - 4 patch sets | 20:53 |
| timburke | WFM! | 20:53 |
| zaitcev | okay | 20:53 |
| zaitcev | I'm working on SLO and I'm tired of being careful not to push a wrong version of bulk by mistake. | 20:54 |
| timburke | yeah, that's fair | 20:55 |
| *** tdasilva has quit IRC | 20:55 | |
| *** NM has quit IRC | 21:07 | |
| *** henriqueof has quit IRC | 21:55 | |
| *** itlinux has joined #openstack-swift | 21:57 | |
| *** zaitcev_ has joined #openstack-swift | 22:19 | |
| *** ChanServ sets mode: +v zaitcev_ | 22:19 | |
| *** zaitcev has quit IRC | 22:22 | |
| *** zaitcev_ has quit IRC | 22:29 | |
| *** zaitcev_ has joined #openstack-swift | 22:41 | |
| *** ChanServ sets mode: +v zaitcev_ | 22:41 | |
| openstackgerrit | Merged openstack/swift master: py3: bulk -- alternate approach https://review.opendev.org/657505 | 22:49 |
| *** itlinux_ has joined #openstack-swift | 22:51 | |
| *** itlinux has quit IRC | 22:52 | |
| *** zaitcev_ has quit IRC | 22:53 | |
| *** zaitcev_ has joined #openstack-swift | 23:05 | |
| *** ChanServ sets mode: +v zaitcev_ | 23:05 | |
| openstackgerrit | Pete Zaitcev proposed openstack/swift master: py3: slo https://review.opendev.org/650507 | 23:12 |
| *** zaitcev_ has quit IRC | 23:19 | |
| *** mvkr has quit IRC | 23:21 | |
| *** gyee has quit IRC | 23:25 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
