| *** sfernand has quit IRC | 00:49 | |
| *** carloss has quit IRC | 03:18 | |
| *** felipe_rodrigues has quit IRC | 03:53 | |
| *** vishalmanchanda has joined #openstack-manila | 04:55 | |
| *** mnaser has quit IRC | 05:59 | |
| *** mnaser has joined #openstack-manila | 06:00 | |
| *** tosky has joined #openstack-manila | 07:46 | |
| openstackgerrit | Maurice Escher proposed openstack/manila master: init_host should be called before RPC consumer is created https://review.opendev.org/c/openstack/manila/+/787321 | 08:16 |
|---|---|---|
| openstackgerrit | Maurice Escher proposed openstack/manila master: init_host should be called before RPC consumer is created https://review.opendev.org/c/openstack/manila/+/787321 | 08:34 |
| *** noonedeadpunk has joined #openstack-manila | 08:37 | |
| noonedeadpunk | hi there! I found our CI failing for manila tempest jobs, with pretty weird reason. I found that in manila-scheduler log http://paste.openstack.org/show/804656/ | 08:39 |
| noonedeadpunk | so in backend we really set `cephfs_enable_snapshots = False` https://zuul.opendev.org/t/openstack/build/01161b92bafe4074a38b1d800eb8f8c4/log/logs/etc/host/manila/manila.conf.txt#32 | 08:39 |
| noonedeadpunk | But I really not sure, if that's a reason for failing scheduler filter | 08:40 |
| noonedeadpunk | I mean, that in case share reports snapshots as supported, but they are disabled in config - that shouldn't be considered as failure. | 08:40 |
| noonedeadpunk | But when it's enabled in config, but reported by share as false - then it should fail | 08:41 |
| noonedeadpunk | or I'm missing smth here? | 08:41 |
| *** lpetrut has joined #openstack-manila | 09:39 | |
| tbarron | noonedeadpunk: as of wallaby that config setting has no effect; cephfs snapshot support is no longer experimental in ceph itself and manila-share passes through the capability to the scheduler | 10:28 |
| tbarron | as in your scheduler log pastebin | 10:28 |
| tbarron | even if you set the deprecated ceph_enable_snapshots option to False | 10:29 |
| noonedeadpunk | well, I'd say it has effect, since scheduling is failing because of that:) | 10:29 |
| tbarron | noonedeadpunk: no it's failing despite that | 10:30 |
| tbarron | the fix is to change your share type | 10:30 |
| tbarron | the scheduler failure is due to mismatch between share type (which is saying gimme a back end w/o snapshot support) and the | 10:30 |
| tbarron | backend itself, which has snapshot support despite the ineffective setting of that deprecated option | 10:31 |
| noonedeadpunk | oh, well, just noticed that `default_share_type` is not valid there... | 10:33 |
| tbarron | noonedeadpunk: manila type-key default-shares set snapshot_support=True | 10:34 |
| tbarron | noonedeadpunk: manila type-key default-shares set create_from_snapshot_support=True | 10:34 |
| tbarron | sub for default-shares | 10:34 |
| tbarron | Then remove that deprecated ineffective option from manila.conf for cleanup purposes as it will confuse. | 10:35 |
| tbarron | noonedeadpunk: This isn't an ideal situation but ceph made a bunch of not particularly backward compatible changes | 10:35 |
| tbarron | though they were good changes | 10:36 |
| tbarron | and we needed to adapt. | 10:36 |
| tbarron | It's in releasenotes of course but ... | 10:36 |
| noonedeadpunk | ok, gotcha. But if we're talking about new manila deployment, do I need to run these commands as well? Or it would be enough just to remove cephfs_enable_snapshots from share at the beggining? | 10:37 |
| noonedeadpunk | Yeah, I saw, but I didn't expect to things fail bacause of that:) | 10:37 |
| tbarron | noonedeadpunk: your expectations are reasonable :( | 10:37 |
| tbarron | noonedeadpunk: on a new deployment you still need a share type that matches the back end capabilities for share creation to succeed | 10:38 |
| tbarron | noonedeadpunk: if osa creates the share type it should create it with those extra spec settings | 10:39 |
| tbarron | noonedeadpunk: as an example, tempest creates a bunch of share types before it runs and it was modified for wallaby in this respect | 10:39 |
| tbarron | manila-tempest-plugin | 10:40 |
| noonedeadpunk | I think we run only these things https://opendev.org/openstack/openstack-ansible-os_manila/src/branch/master/tasks/manila_backends.yml#L39-L40 | 10:41 |
| noonedeadpunk | ok, I got the direction, will try to check this now) | 10:41 |
| tbarron | noonedeadpunk: you need a couple more items 'snapshot_support: True' 'create_share_from_snapshot_support: True' | 10:46 |
| tbarron | noonedeadpunk: Thanks, and sorry for the hassle. | 10:46 |
| noonedeadpunk | but it's only in case share_driver is CephFSDriver ? | 10:47 |
| tbarron | noonedeadpunk: the principle holds for all back ends but only the CephFSDriver had such a change in wallaby | 10:52 |
| noonedeadpunk | well, what I'm trying to understand is when to run these commands... | 10:53 |
| noonedeadpunk | because I think you might not always want to enable snapshots? | 10:53 |
| tbarron | The share type setup is one time, post deployment, prior to use. | 10:54 |
| noonedeadpunk | meant when in terms of scenarios :) | 10:54 |
| tbarron | well manila back ends normally have snapshot capability. It was a deficiency on cephs part not to match. To be clear | 10:55 |
| noonedeadpunk | aha, so I can jsut run that unconditionally let's say? | 10:56 |
| tbarron | the share owner still has to issue 'manila snapshot create ...' for snapshots to be created. this isn't a run-shapshots-automatically off the clock thing. | 10:56 |
| tbarron | If a user doesn't want any snapshots, doesn't want the extra disk consumption or whatever (after share and snap diverge over time) then they just don't create them. | 10:57 |
| tbarron | or they delete older ones. | 10:57 |
| gouthamr | "because I think you might not always want to enable snapshots?" --> these extra-specs are optional: if you as an administrator don't want your users to create snapshots or clone snapshots, you shouldn't set snapshot_support and create_share_from_snapshot_support in the share type | 10:57 |
| tbarron | gouthamr: what are you doing up? | 10:58 |
| noonedeadpunk | ok, yeah, agree. Was just trying to understand if there're cases where this might be smth you might want to avoid from setting | 10:58 |
| gouthamr | didn't mean to interrupt your discussion :P | 10:58 |
| gouthamr | i wanted to say the failures here are purely manila-tempest-plugin's inflexibility in one sense | 10:59 |
| noonedeadpunk | gouthamr: well, everything started from that when I don't set this, scheduler fails to select host for creating share | 10:59 |
| noonedeadpunk | http://paste.openstack.org/show/804656/ | 10:59 |
| gouthamr | yep | 11:00 |
| noonedeadpunk | so I kind of can't not setting snapshot_support if I don't want users to use snapshots? | 11:01 |
| *** carloss has joined #openstack-manila | 11:01 | |
| gouthamr | our tempest tests lack the ability to have a "don't care" situation with these extra-specs - they default to setting "snapshot_support" and "create_share_from_snapshot_support" into share types | 11:02 |
| gouthamr | that's what's failing in your test jobs | 11:02 |
| noonedeadpunk | well, yes. I can kind of make our tests match tempest, but was just more wondering about if it should work overall in non-ci or dunno | 11:04 |
| noonedeadpunk | (eventually we're running tempest as well) | 11:04 |
| gouthamr | oh i was under the impression you're running manila-tempest-plugin tests | 11:04 |
| gouthamr | what are you running? :) | 11:05 |
| gouthamr | i mean is this a custom test that i can see somewhere? | 11:05 |
| noonedeadpunk | ah, sorry | 11:06 |
| noonedeadpunk | I read your statement wrong way) | 11:06 |
| noonedeadpunk | yes, we're running tempest scenario | 11:06 |
| noonedeadpunk | but it used to have `run_snapshot_tests` option? | 11:06 |
| gouthamr | it still does - and that defaults to True, unless you're overriding it | 11:07 |
| noonedeadpunk | we have it configured that way https://opendev.org/openstack/openstack-ansible/src/branch/master/tests/roles/bootstrap-host/templates/user_variables_manila.yml.j2#L35-L57 | 11:07 |
| gouthamr | i see, ty | 11:07 |
| noonedeadpunk | (and it's false for this scenario) | 11:07 |
| gouthamr | and these tests are only being run with CephFS? (i assume yes, based on the manila_backends section at the bottom of that yaml) | 11:08 |
| noonedeadpunk | yeah | 11:09 |
| noonedeadpunk | we're trying just to do minimal set of tests to ensure that deployment basicaly working | 11:09 |
| gouthamr | awesome - then for Wallaby and beyond, you can set "run_snapshot_tests=True" (or remove the override) | 11:09 |
| noonedeadpunk | and rely on massive testing you perform | 11:09 |
| gouthamr | i see, if you don't want to run snapshot tests, you can set "run_snapshot_tests=False" and set "capability_snapshot_support=True" | 11:10 |
| noonedeadpunk | but I should add running `manila type-key default-shares set snapshot_support=True` and `manila type-key default-shares set create_from_snapshot_support=True` anyway? | 11:10 |
| gouthamr | but i wouldn't recommend it | 11:10 |
| noonedeadpunk | aha, gotcha | 11:10 |
| gouthamr | noonedeadpunk: this is what's tripping us up here: https://github.com/openstack/manila-tempest-plugin/blob/602f101c14ea9130226359942636fb5bf0c9e782/manila_tempest_tests/tests/scenario/manager_share.py#L459-L464 | 11:14 |
| gouthamr | noonedeadpunk: we're creating a share type and setting "snapshot_support" from the config opt "capability_snapshot_support=True", which defaults to "run_snapshot_tests" when not set | 11:15 |
| gouthamr | i was exploring some cleanup in the tests: https://review.opendev.org/c/openstack/manila-tempest-plugin/+/785307 and i missed these scenario tests | 11:17 |
| gouthamr | so thanks for pointing this out, i'll include it in my cleanup - but, for your immediate relief, you can set "capability_snapshot_support=True" in https://opendev.org/openstack/openstack-ansible/src/branch/master/tests/roles/bootstrap-host/templates/user_variables_manila.yml.j2#L35-L57 and still have the right thing happening | 11:18 |
| noonedeadpunk | cool awesome, thanks for pointing to that setting! | 11:19 |
| *** dviroel_ has quit IRC | 11:19 | |
| openstackgerrit | Maurice Escher proposed openstack/manila master: init_host should be called before RPC consumer is created https://review.opendev.org/c/openstack/manila/+/787321 | 11:29 |
| noonedeadpunk | gouthamr: umm.. but in https://github.com/openstack/manila-tempest-plugin/blob/602f101c14ea9130226359942636fb5bf0c9e782/manila_tempest_tests/tests/scenario/manager_share.py#L459-L464 it's using `CONF.share.capability_snapshot_support` or I read it wrong? | 11:44 |
| gouthamr | noonedeadpunk: https://github.com/openstack/manila-tempest-plugin/blob/602f101c14ea9130226359942636fb5bf0c9e782/manila_tempest_tests/plugin.py#L39-L53 | 11:47 |
| gouthamr | noonedeadpunk: if CONF.share.capability_snapshot_support isn't set, we default to the value of CONF.share.run_snapshot_tests | 11:48 |
| noonedeadpunk | I think I just can't understand why setting everything to false is not going to work... | 11:49 |
| noonedeadpunk | because driver will explicitly say that it's supported? | 11:50 |
| noonedeadpunk | and that's what not handled I guess | 11:50 |
| noonedeadpunk | ok, gotcha | 11:50 |
| noonedeadpunk | I just though that if I wan't run `manila type-key default-shares set snapshot_support=True` driver won't report it as supported | 11:51 |
| noonedeadpunk | and we don't run this bit at the moment | 11:51 |
| gouthamr | ah, nope - the cephfs driver reports capability as being True since Wallaby, and you can't control that... | 11:54 |
| gouthamr | since Victoria* | 11:54 |
| gouthamr | in the past, since CephFS snapshots were experimental (a ceph side limitation) it made sense to have a configuration option to allow administrators to opt in | 11:55 |
| *** sfernand has joined #openstack-manila | 12:57 | |
| openstackgerrit | Liron Kuchlani proposed openstack/manila-tempest-plugin master: Stop supporting microversion 2.23 and less https://review.opendev.org/c/openstack/manila-tempest-plugin/+/781074 | 13:26 |
| openstackgerrit | Liron Kuchlani proposed openstack/manila-tempest-plugin master: Stop supporting microversion 2.23 and less https://review.opendev.org/c/openstack/manila-tempest-plugin/+/781074 | 13:33 |
| openstackgerrit | Dina Saparbaeva proposed openstack/manila-ui master: WIP: API v2.45, adds metadata support to access rules https://review.opendev.org/c/openstack/manila-ui/+/783041 | 13:50 |
| openstackgerrit | Tom Barron proposed openstack/manila master: Fix api error message for snapshot creation failure https://review.opendev.org/c/openstack/manila/+/787371 | 13:52 |
| openstackgerrit | Liron Kuchlani proposed openstack/manila-tempest-plugin master: Setup snapshot extra specs only if required https://review.opendev.org/c/openstack/manila-tempest-plugin/+/785307 | 13:57 |
| gouthamr | Hello folks! We're beginning in https://www.openstack.org/ptg/rooms/austin in just a minute! | 13:59 |
| *** lpetrut has quit IRC | 14:29 | |
| noonedeadpunk | sorry for bothering you today, but any guess wtf might be wrong here? https://zuul.opendev.org/t/openstack/build/a6aa7c2a74674d7694812f6d15441137/log/logs/host/manila-api.service.journal-13-32-09.log.txt#250 ? | 14:49 |
| *** dviroel is now known as dviroel|lunch | 15:34 | |
| gouthamr | noonedeadpunk: ouch, haven't seen that before | 16:03 |
| gouthamr | lemme take a look after the PTG session today, any chance you can report a bug? | 16:04 |
| gouthamr | is it possible the version of oslo_context is too old? | 16:04 |
| *** tkajinam has quit IRC | 16:07 | |
| *** tkajinam has joined #openstack-manila | 16:07 | |
| noonedeadpunk | gouthamr: we're taking it from u-c, so unlikely... Well, np, I will try to launch a sandbox and see what's going on there... | 16:08 |
| noonedeadpunk | jsut thought maybe smth you know from top of your head) | 16:08 |
| noonedeadpunk | as eventually I think that worked in the morning... | 16:09 |
| *** vhari has quit IRC | 16:26 | |
| *** vhari has joined #openstack-manila | 16:29 | |
| *** dviroel|lunch is now known as dviroel | 16:30 | |
| gouthamr | noonedeadpunk: sorry late ack, nope nothing from the top of my head - although i did see a note about _read_deleted being unset in the context object from some nova code: https://opendev.org/openstack/nova/src/branch/master/nova/context.py#L163-L179 | 18:12 |
| gouthamr | or not, we have getters/setters for this code here in manila: https://opendev.org/openstack/manila/src/branch/master/manila/context.py#L73-L86 | 18:14 |
| *** jmlowe has joined #openstack-manila | 18:44 | |
| *** vishalmanchanda has quit IRC | 18:54 | |
| openstackgerrit | Dina Saparbaeva proposed openstack/manila-ui master: WIP: API v2.45, adds metadata support to access rules https://review.opendev.org/c/openstack/manila-ui/+/783041 | 19:06 |
| *** lpetrut has joined #openstack-manila | 19:32 | |
| *** lpetrut has quit IRC | 19:34 | |
| *** tosky has quit IRC | 23:42 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!