| timburke_ | andybotting, proxyfs is still a thing, but i'm not entirely sure of its development state. i know it's moved homes -- now at https://github.com/NVIDIA/proxyfs -- and my understanding is that it's moving away from trying to enable bimodal access via proxy-server middleware toward having a local FUSE-based client that just happens to use swift as a storage backend | 00:12 |
|---|---|---|
| timburke_ | no idea about the state of keystone support there; my assumption is that no one has tried it, but even that i don't know for certain | 00:13 |
| andybotting | A FUSE model sounds perfect for us | 00:16 |
| timburke_ | fwiw, other things you might look into are https://github.com/kahing/goofys, https://github.com/dask/s3fs/, or https://rclone.org/commands/rclone_mount/ -- though i don't have much of any experience with any of them | 00:16 |
| andybotting | I remember seeing OVH had something they wrote that looked similar too, but they say it's not being developed any more | 00:17 |
| andybotting | What we would really like to do is create a keystone application credential (on behalf of the user) and inject that into their VM and automatically mount their swift storage | 00:18 |
| andybotting | I'm not sure if the S3 interface would be practical for that workflow, but I haven't looked much into it ether | 00:19 |
| timburke_ | andybotting, out of curiosity, what all requirements do you have? will the volume need to be r/w, or could it be read-only? will multiple clients need to be able to mount the same volume, or will it be a 1:1 mapping? | 00:19 |
| andybotting | We need r/w but expect that it would be written to from one source at a time | 00:20 |
| andybotting | Its for a virtual desktop service. Users will have one desktop at a time, but would mount swift as their way of getting data in/out | 00:21 |
| andybotting | Then they could use that same storage for other services too | 00:21 |
| andybotting | But the I expect the usage of it would be more for shuffling data in and out, rather than doing processing on directly | 00:22 |
| timburke_ | makes sense; have local scratch space and use swift for inputs/outputs | 00:23 |
| andybotting | that's it | 00:23 |
| andybotting | goofys looks pretty neat | 00:24 |
| timburke_ | rclone mount might be worth checking out, too, if you'd rather not add in the s3api complexity | 00:25 |
| timburke_ | it should have native swift support | 00:25 |
| andybotting | yeah, just looking now | 00:26 |
| timburke_ | for all these things, i expect the vfs cache tuning is going to be key in determining whether you can get acceptable performance | 00:27 |
| andybotting | Yeah. I think we'll need to make sure the users have a realistic expectation of what this storage will be for | 00:27 |
| andybotting | s3api might be ok, if I (as an admin) can generate a credential on the user's behalf | 00:29 |
| andybotting | which it looks like I can :) | 00:30 |
| andybotting | I'll have a play with these and see how I go. Thanks timburke_ | 00:33 |
| timburke_ | andybotting, no prob, happy to help! good luck! | 00:34 |
| mor3s | not sure how it happened but now i have messages on ring.gz being obsolete "Ring file account.ring.gz is obsolete | 01:48 |
| mor3s | " .. how do i resolve this? | 01:48 |
| kota | mor3s: it seems like your builder file have new changes that is not dumped to the ring file yet | 04:51 |
| kota | mor3s: you can see your ring balance via `swift-ring-builder <builder file>` and can check the balance is now as you expected. | 05:00 |
| mor3s | the rebalance was done long time ago | 05:03 |
| kota | usually, `swift-ring-builder <builder file> rebalance` will create the newer balanced ring.gz file, then the message should be changed to "Ring file account.ring.gz is up-to-date" | 05:03 |
| kota | mor3s: https://gist.github.com/bloodeagle40234/9c44f3a7bab47a2327364ccf6577f010 <- is an example for you to get swift-ring-file builder behaviro | 05:08 |
| kota | behavior | 05:08 |
| mor3s | does it sounds like someone made changes to the builder files (eg add devices) and forgot to run the rebalance command? | 05:10 |
| kota | it's... likely that builder file is staging state, then ring file is the committed one. the way to commit from the builder file to the ring is either `rebalance` command or `write_ring` IIRC. | 05:10 |
| mor3s | can we see the content of gz files? | 05:11 |
| kota | ah... the way I remember is using swift's Ring class but... it's not from command line interface... | 05:12 |
| kota | let me look for the way... 1 sec | 05:12 |
| kota | hmm... not found | 05:14 |
| kota | one way to recover the builder file from the ring file (i.e. convert ring to builder) but it should not ensure the complete builder file IIRC. | 05:15 |
| kota | mor3s: I'm not sure, it's that you are looking for but https://gist.github.com/bloodeagle40234/0670f4ff390ee1049efb2bf021816bac is one way to extract ring file data to meaningful instance. | 05:18 |
| mor3s | let me take a look at it... thanks kota | 05:19 |
| kota | but usually it's an instance with iterator of nodes and partition tables so it may be hard to get difference with your builder file without deeper swift knowledge | 05:19 |
| timburke__ | might try something like https://gist.github.com/matthewoliver/ce5172dbf1bcf8143184686517ad67ca -- see mattoliver's comment a few years ago: https://meetings.opendev.org/irclogs/%23openstack-swift/%23openstack-swift.2018-01-19.log.html#t2018-01-19T03:55:41 | 14:44 |
| timburke__ | mor3s, ^^^ | 14:44 |
| opendevreview | Andre Aranha proposed openstack/swift master: WIP/DNM: Add FIPS CI jobs https://review.opendev.org/c/openstack/swift/+/796057 | 17:02 |
| opendevreview | Andre Aranha proposed openstack/swift master: WIP/DNM: Add FIPS CI jobs https://review.opendev.org/c/openstack/swift/+/796057 | 17:06 |
| opendevreview | Andre Aranha proposed openstack/swift master: WIP/DNM: Add FIPS CI jobs https://review.opendev.org/c/openstack/swift/+/796057 | 18:03 |
| opendevreview | Andre Aranha proposed openstack/swift master: WIP/DNM: Add FIPS CI jobs https://review.opendev.org/c/openstack/swift/+/796057 | 18:04 |
| reid_g | Anybody good at integrating keystone auth w/swift? I am able to generate my token with `swift auth` but any other swift commands are failing with 401. | 18:16 |
| DHE | I'm doing it. It's pretty easy if you have the URL and auth token... curl -H "X-Auth-Token: $OS_AUTH_TOKEN" $OS_STORAGE_URL/$containername/$objectname" -o saved-object-name.ext | 18:33 |
| DHE | that would save the indicated $objectname from $container to a local file | 18:33 |
| timburke_ | reid_g, sounds like an issue in the [filter:keystoneauth] section in proxy-server.conf -- what keystone roles does the user have for the project? what roles are listed for operator_roles and reseller_admin_role? does the reseller_prefix match how you set up the storage endpoint in keystone? | 18:36 |
| reid_g | In Keystone I created a role called osg_user and my user is assigned to that. In my swift-proxy.conf I have operator_roles=admin, osg_user. I did not specify a reseller_admin_role but the resller_prefix=AUTH_ | 18:43 |
| reid_g | My OS_STORAGE_URL=http://swiftproxy:8080/v1/AUTH_eead7736f5374734b99b9294d06abcfb | 18:44 |
| DHE | [filter:keystoneauth] use = egg:swift#keystoneauth operator_roles = admin, swiftoperator | 19:52 |
| DHE | this is what i have. users with swift access have the swiftoperator role, so I can split it off from virtual machine stuff in openstack | 19:52 |
| reid_g | maybe dumb question. Using puppet-swift and it calls it keystone in pipline and creates a section called [filter:keystone] use=egg:swift#keystoneauth in swift-proxy.conf. Does that wording matter? | 20:11 |
| DHE | I don't think so. it has the correct use= value which should be the important part | 20:20 |
| timburke_ | reid_g, huh. proxy conf seems right then -- might try turning up the log level, see what extra info you can get out of it. sorry, it's been a bit since i last set up swift+keystone, so i don't entirely remember | 20:38 |
| timburke_ | the different filter name is definitely fine, though -- important bits are having the name in the pipeline and the name in the [filter:...] section match, and having the use line right | 20:39 |
| timburke_ | sanity check -- what's the pipeline order? auth_token is left of keystone, yeah? | 20:40 |
| reid_g | Yes "authtoken s3api s3token keystone" | 20:41 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!