Thursday, 2015-05-28

*** mtanino has quit IRC00:09
*** haomaiwang has quit IRC00:30
*** esker has joined #openstack-manila00:54
*** tbarron has quit IRC01:37
*** tbarron has joined #openstack-manila01:38
*** BitSmith has joined #openstack-manila02:03
*** haomaiwang has joined #openstack-manila02:24
openstackgerritMerged openstack/manila: Make devstack install manila-ui if horizon is enabled  https://review.openstack.org/18477802:27
*** vbellur has joined #openstack-manila02:31
openstackgerritMerged openstack/manila: glusterfs: Edit doc and comments  https://review.openstack.org/18530902:35
*** BitSmith has quit IRC02:36
*** zhonghua-lee has joined #openstack-manila02:51
openstackgerritzhongjun proposed openstack/manila: Huawei manila driver code refactoring  https://review.openstack.org/17663703:10
*** zaitcev has quit IRC03:46
*** lpetrut has joined #openstack-manila03:46
*** haomaiwang has quit IRC03:57
*** zhonghua-lee has quit IRC04:00
*** haomaiwang has joined #openstack-manila04:01
*** xyang1 has quit IRC04:11
*** bswartz has quit IRC04:28
*** rcallawa has quit IRC04:28
*** sgotliv has joined #openstack-manila04:30
*** lpetrut has quit IRC04:39
*** haomaiwang has quit IRC04:39
*** cknight has quit IRC04:49
*** openstackgerrit has quit IRC04:50
*** openstackgerrit has joined #openstack-manila04:50
openstackgerritMerged openstack/python-manilaclient: Drop incubating theme from docs  https://review.openstack.org/18619904:55
*** nkrinner has joined #openstack-manila04:59
*** lpetrut has joined #openstack-manila05:24
openstackgerritMerged openstack/manila: Drop incubating theme from docs  https://review.openstack.org/18619605:26
openstackgerritMerged openstack/manila: Remove ServiceClient from share_client  https://review.openstack.org/18515205:27
*** vbellur has quit IRC05:44
*** haomaiwa_ has joined #openstack-manila05:56
*** vbellur has joined #openstack-manila06:23
*** lpetrut has quit IRC06:38
*** lpetrut has joined #openstack-manila07:02
*** lpetrut has quit IRC07:11
*** lpetrut has joined #openstack-manila07:22
*** zhonghua-lee has joined #openstack-manila07:27
*** Maike has joined #openstack-manila07:29
*** u_glide has joined #openstack-manila07:46
openstackgerritIgor Malinovskiy proposed openstack/manila: Implement extend_share() method in Generic driver  https://review.openstack.org/18238307:48
openstackgerritIgor Malinovskiy proposed openstack/manila: Implement tempest tests for share extend API  https://review.openstack.org/18349707:49
openstackgerritIgor Malinovskiy proposed openstack/manila: Implement tempest tests for share extend API  https://review.openstack.org/18349707:52
openstackgerritIgor Malinovskiy proposed openstack/manila: Add share shrink API  https://review.openstack.org/18406907:55
openstackgerritIgor Malinovskiy proposed openstack/python-manilaclient: Add share shrink API  https://review.openstack.org/18538008:06
openstackgerritli,chen proposed openstack/manila: Implement versioned object for share export locations  https://review.openstack.org/17369208:34
*** kaisers has joined #openstack-manila08:37
MaikeHi, I'm using devstack kilo on Ubuntu 14.04 LTS. Where can I change the setting that configures which manila service image should be used by manila?08:37
*** haomaiwa_ has quit IRC08:39
vponomaryovMaike: https://github.com/openstack/manila/blob/770c272b/devstack/plugin.sh#L8908:40
vponomaryovMaike: in link provided list of env vars that you can redefine and do it in the same way as all other devstack config changes you do - using localrc file08:41
Maikevponomaryov: okay, thanks. I already found this part, but it doesn't worked... I will try it again08:43
vponomaryovMaike: how exactly id does not work?08:44
vponomaryovs/id/it/08:44
*** 7YUAAETE8 has joined #openstack-manila08:45
*** mkoderer has quit IRC08:50
*** mkoderer has joined #openstack-manila08:54
*** rraja has joined #openstack-manila08:58
*** lpetrut1 has joined #openstack-manila09:13
*** lpetrut has quit IRC09:14
*** Triveni has joined #openstack-manila09:19
*** 7YUAAETE8 has quit IRC09:27
*** rraja_ has joined #openstack-manila09:55
*** rraja has quit IRC09:57
openstackgerritzhongjun proposed openstack/manila: Huawei manila driver code refactoring  https://review.openstack.org/17663709:57
*** rraja_ has quit IRC09:58
*** rraja has joined #openstack-manila10:00
*** zhonghua-lee has quit IRC10:22
*** Triveni has quit IRC10:49
*** rcallawa has joined #openstack-manila10:57
*** rcallawa has quit IRC11:00
*** rcallawa has joined #openstack-manila11:01
*** rcallawa_ has joined #openstack-manila11:05
*** rcallawa has quit IRC11:09
*** rcallawa_ has quit IRC11:18
*** rcallawa has joined #openstack-manila11:19
*** zhonghua-lee has joined #openstack-manila11:22
*** timcl has joined #openstack-manila11:27
*** haomaiwang has joined #openstack-manila11:28
*** openstackgerrit has quit IRC11:39
*** openstackgerrit has joined #openstack-manila11:39
*** haomaiwang has quit IRC11:40
*** marcusvrn1 has joined #openstack-manila11:52
*** marcusvrn has quit IRC11:55
*** ganso_ has joined #openstack-manila11:55
*** zhonghua-lee has quit IRC11:56
*** rcallawa has quit IRC11:56
*** akerr has joined #openstack-manila12:04
*** akerr_ has joined #openstack-manila12:05
openstackgerritJulia Varlamova proposed openstack/manila: Share_server-pool mapping  https://review.openstack.org/18045012:06
openstackgerritJulia Varlamova proposed openstack/manila: Add PoolFilter for Manila scheduler  https://review.openstack.org/18405312:06
*** rraja has quit IRC12:08
*** akerr has quit IRC12:08
*** zhonghua-lee has joined #openstack-manila12:24
*** rcallawa has joined #openstack-manila12:39
*** kaisers has quit IRC12:40
*** kaisers has joined #openstack-manila12:41
*** rraja has joined #openstack-manila12:47
*** bswartz has joined #openstack-manila12:48
*** zhonghua-lee has quit IRC12:57
openstackgerritValeriy Ponomaryov proposed openstack/manila: Remove unused attr status from models  https://review.openstack.org/18637413:02
openstackgerritValeriy Ponomaryov proposed openstack/manila: Remove unused attr status from models  https://review.openstack.org/18637413:08
*** akshai has joined #openstack-manila13:09
openstackgerritValeriy Ponomaryov proposed openstack/manila: Remove unused attr status from models  https://review.openstack.org/18637413:09
*** esker has quit IRC13:12
*** Maike_ has joined #openstack-manila13:14
*** bswartz has quit IRC13:16
*** Maike has quit IRC13:18
*** bswartz has joined #openstack-manila13:22
*** akerr_ has quit IRC13:24
*** dustins has joined #openstack-manila13:29
*** timcl has quit IRC13:46
*** vbellur has quit IRC13:52
*** xyang1 has joined #openstack-manila13:57
*** nkrinner has quit IRC13:59
*** Maike__ has joined #openstack-manila14:01
*** timcl has joined #openstack-manila14:03
*** xyang1 has quit IRC14:04
*** vkmc has joined #openstack-manila14:05
*** Maike_ has quit IRC14:05
*** rushil has joined #openstack-manila14:07
*** rushil has quit IRC14:08
*** cknight has joined #openstack-manila14:13
*** rushil has joined #openstack-manila14:18
*** mtanino has joined #openstack-manila14:28
*** Maike__ has quit IRC14:33
*** xyang1 has joined #openstack-manila14:35
*** bswartz has quit IRC14:47
*** bswartz has joined #openstack-manila14:47
*** esker has joined #openstack-manila14:48
*** markstur has quit IRC14:53
*** Zhongjun has joined #openstack-manila14:54
*** markstur has joined #openstack-manila14:55
*** mtanino has quit IRC15:06
*** mtanino has joined #openstack-manila15:07
*** mtanino has quit IRC15:07
*** mtanino has joined #openstack-manila15:08
*** fthiagogv has joined #openstack-manila15:16
*** vbellur has joined #openstack-manila15:21
*** timcl has quit IRC15:25
*** akerr has joined #openstack-manila15:27
*** timcl has joined #openstack-manila15:30
*** akerr has quit IRC15:35
*** timcl has quit IRC15:35
*** akerr has joined #openstack-manila15:35
*** fthiagogv has quit IRC15:37
*** timcl has joined #openstack-manila15:37
*** fthiagogv has joined #openstack-manila15:37
*** chlong has quit IRC15:54
*** mtanino_ has joined #openstack-manila16:02
*** mtanino has quit IRC16:03
*** absubram has joined #openstack-manila16:05
*** rraja has quit IRC16:07
*** chlong has joined #openstack-manila16:07
openstackgerritValeriy Ponomaryov proposed openstack/manila: Remove unused attr status from models  https://review.openstack.org/18637416:07
*** Triveni has joined #openstack-manila16:17
openstackgerritValeriy Ponomaryov proposed openstack/manila: Fix policy check for API 'security service update'  https://review.openstack.org/18646016:22
*** Triveni has quit IRC16:27
*** lpetrut1 has quit IRC16:28
*** haomaiwang has joined #openstack-manila16:29
*** Triveni has joined #openstack-manila16:45
*** Triveni has quit IRC16:52
*** akshai has quit IRC16:56
*** akshai has joined #openstack-manila16:58
*** lpetrut has joined #openstack-manila17:01
*** rcallawa_ has joined #openstack-manila17:02
*** rushil has quit IRC17:02
*** akshai has quit IRC17:04
*** akshai has joined #openstack-manila17:05
*** rcallawa has quit IRC17:05
*** rushil has joined #openstack-manila17:05
*** zaitcev has joined #openstack-manila17:30
*** sage has quit IRC17:31
*** sage has joined #openstack-manila17:32
*** sage has quit IRC17:48
bswartzganso_: ping17:55
ganso_bswartz: pong17:59
bswartzganso_: I replied to your ML post17:59
bswartzI wanted to know about your proposal to implement migration without creating a second DB row17:59
bswartzsince that's new to me17:59
*** akshai has quit IRC18:00
*** vbellur has quit IRC18:01
ganso_bswartz: I will reply in detail to the ML, but in summary, Valeriy suggested not creating a temporary DB entry, so the ID used would always be the original one18:01
*** rcallawa_ has quit IRC18:01
bswartzganso_: here is fine also18:02
ganso_vponomaryov: ^18:02
*** rcallawa has joined #openstack-manila18:02
bswartzor a google hangout if that would be easier for you18:02
bswartzI'm really curious though18:02
ganso_ok, we can discuss it here18:02
ganso_the original approach was based on Cinder's: create a totally new volume in the destination backend, new API ID, new physical volume ID18:03
ganso_then after copying data, copy all DB properties from the new DB entry to the original one18:03
bswartzyes18:03
ganso_so the original API ID would be linked to the physical share ID, because the physical share ID is a property value18:04
ganso_that is transferable18:04
bswartzthe advantage to that approach is that drivers don't need to change at all for it to work in the fallback case18:04
ganso_yes18:04
*** rushil has quit IRC18:04
bswartzso what is the new proposal18:04
*** akshai has joined #openstack-manila18:05
ganso_take by example that the generic driver is storing the volume_id in the private driver storage18:05
ganso_the volume_id in this case is the physical share ID I mentioned above18:06
ganso_when we create the new share in the destination backend, we would store that value in the private driver storage18:06
ganso_since private driver storage is restricted by backends, it will not conflict with the source backend private driver storage18:07
bswartzbut you can't create a new share without creating a DB row first18:07
ganso_I thought so too at first, I am still analyzing that, and so far, it looks to me that we can do so by providing the source DB entry, and skip the API call18:07
bswartzunless we make a new driver entry point called "create_migration_destination_share" which creates a share that has no DB ro18:07
bswartzrow18:07
ganso_go straight to the manager18:07
bswartzthen you'd have 2 drivers thinking they owned the same share18:08
ganso_but it is a little dangerous, I am still analyzing that, it would be complicated to cleanup error states18:08
ganso_yes18:08
bswartzI don't think that's a smart approach18:09
bswartzthe error cases would be very hard to deal with18:09
ganso_yes18:09
bswartzif you think about either or both backends being bounced in the middle of the migration operation18:09
ganso_using the temporary Cinder approach is much easier to handle18:09
bswartzI would be less scared if we made it an entirely separate driver call, but then we wouldn't have a universal fallback approach for share migration18:10
ganso_a separate call would be more secure, indeed18:10
ganso_and it would be related to the universal fallback approach18:10
ganso_the driver migration approach would be before that18:11
*** rushil has joined #openstack-manila18:11
bswartzwere the other 2 approaches discussed?18:11
ganso_if the driver is unsuccessful migrating, then we proceed to generic migration18:11
ganso_which is the approach based on Cinder's18:11
ganso_driver migration may not even create other DB entries, it is up to driver18:11
ganso_in fact, if I am not mistaken, the driver is not allowed to do that18:12
bswartzbut you can't do a generic migration without either (1) doing the scary thing you proposed above or (2) forcing all drivers to implement a new method or (3) creating 2 DB rows18:12
ganso_what do you mean by (2)?18:13
*** sage has joined #openstack-manila18:13
bswartzcorrect, the driver cannot create new DB entries -- the theory would be that the scheduler/API would create the new row and then tell the driver to try to do optimized migration from one share to the other18:13
bswartzby (2) I mean make drivers implement something like "create_migration_destination_share" so the driver knows that it's creating a migration destination instead of a normal share and18:14
bswartzs/and//18:14
ganso_not really, the way I see the driver would not really need to interact with the scheduler/API, driver migration is when it migrates from vendor A backend to another vendor A backend, so it can really do it in background, at the backend level, and just update the export location18:15
bswartzin case it's not obvious I really like the elegance of the cinder approach to migration -- I just think they mishandled the ID swap approach18:16
bswartzganso_: let me understand your proposal better then18:17
openstackgerritValeriy Ponomaryov proposed openstack/manila: Transform share and share servers statuses to lowercase  https://review.openstack.org/18652018:17
bswartzwhat decides where the destination share will go?18:17
bswartzsuppose I have a share on backend A, which is a netapp18:17
ganso_in (2), my original idea was to have the manager implement the "create_migration_destination_share" and call the driver's "create_share", but since the driver will have total control over the private storage, then it may have to be a different method, I am still looking at that18:17
bswartzand the administrator says: migrate to backend B, which is also netapp18:17
bswartzand what if the administrator says: migrate to backend C, which is HDS?18:18
bswartzwhat would the Host field in the DB for that share say?18:19
ganso_in your first scenario, the "migrate_share" method starts, calls "driver.migrate_share" and it returns a model update, done18:19
bswartzcalls that method on which backend?18:20
ganso_in your second scenario, from netapp to HDS, the "migrate_share" manager method starts", calls "driver.migrate_share", which returns None, and the code proceeds to generic migration, which we are still debating how to properly handle the IDs18:21
ganso_the source backend18:21
bswartzganso_: and how does the source backend know where it's supposed to go?18:21
bswartzhold on I want to stay on the first example before going to the second18:22
ganso_I haven't really looked how Cinder drivers implement their own migration18:23
ganso_hold on let me grab a line of code18:23
bswartzit's just that if we never store the destination of the migration anywhere, I'm confused about how the source backend knows where to try to migrate to and how we recover from a crash before the migration is done18:24
ganso_https://github.com/openstack/cinder/blob/master/cinder/volume/manager.py#L142018:24
ganso_it has the host parameter18:24
ganso_which is the destination host18:25
ganso_I remember taking a quick look somewhere that there are ways for driver to obtain host information based on that string value18:25
bswartzganso_: ugh18:25
bswartzI didn't realize cinder did that for the optimized case18:26
bswartzdoes anyone implement that codepath?18:26
*** dustins has quit IRC18:26
bswartzbecause I think it's a giant bug waiting to happen18:26
ganso_I am not sure, I may have seen one vendor implementing that when I first started studying it, after our previous midcycle meeting18:27
ganso_I can look it up again18:27
bswartzclearly in cinder very different things happen with optimized and non-optimized volume migration18:27
bswartzI kind of think that's bad by itself18:27
ganso_the way I understood is that18:28
ganso_let's say my backend is HDS18:28
bswartzI would rather have the 2 codepaths as similar as possible, with the optimization just being how the data gets from the source to the destination18:28
ganso_and it is performing migration to another HDS backend18:28
ganso_the source backend would talk to the other backend inside that method, like, issue commands to create the share in the destination backend, because HDS drivers know the protocol to talk to each other18:29
ganso_the way I see it, this is the optimized approach18:29
*** Zhongjun has quit IRC18:29
bswartzyou mean one driver instance talks to 2 storage controllers18:29
bswartznot 2 drivers talking to eachother18:29
ganso_yes18:29
ganso_like a cluster array18:30
bswartzI agree that's the right way to handle the data movement18:30
bswartzMy issues are around the database/state-trackinf side of this18:30
bswartztracking18:31
ganso_the database would not be touched in that driver migration approach18:31
bswartzright18:31
ganso_because the model update returned would only update the export location18:31
bswartzso if anything at all goes wrong, you completely forget it ever happened18:31
ganso_so only the physical share moved18:31
bswartzwell not anything at all18:31
ganso_in fact, it updates the export location and the host columns18:31
bswartzbut if the c-vol instance doing the migration dies before the model update is returned then we forget about the migration attempt18:32
ganso_yes, this is something VERY problematic that happens on the Cinder migration as well18:32
ganso_every time an error occurs, we never know if migration in fact succeeded18:32
ganso_because the error state is cleaned, and the original ID remains, with status available18:33
bswartzthat the main reason I'm in favor of creating a new row immediately and putting the original share into some kind of migration state18:33
bswartzthat way if anything happens after that, we can pick up where we left off, or go back and clean up the error18:33
ganso_yes, it has been discussed in the Cinder migration design session18:33
ganso_we have a field for that18:33
ganso_for cleaning up18:34
ganso_but at the design summit we discussed having a "last migration" field18:34
ganso_something like that18:34
bswartzokay so let's back up for a minute18:34
bswartzforget about the optimized case and go to the non-optimized case18:35
ganso_but thinking more about it, I think the host field is enough to say where the share really is18:35
ganso_ok18:35
bswartzI think if we can solve that one, then how we handle optimized migrations will be just a small detail18:35
bswartzso what happens in the A->C case (netapp to hds)18:35
ganso_original idea: create a totally new share in C (physical + DB), copy data over, delete physical share in A, copy DB entries, and delete the temporary DB entry. End result is original ID pointing to physical share at C18:37
bswartzright..18:37
ganso_that is the one that is prototyped18:38
bswartzI was comfortable with that approach18:38
ganso_me too18:38
bswartzalthough we would want to decide on using 2 ID fields or allowing re-IDing of the share18:38
ganso_everything that is being discussed, about private driver storage, are possible improvements18:39
bswartzthat's a minor detail compared to not creating a new row at all though18:39
bswartzso what is the new idea that doesn't involve creating a new row?18:39
*** xyang1 has quit IRC18:40
ganso_the idea was to create a new share in destination backend, either by a separate method or providing the original ID, the driver would store the physical share ID in the private driver storage, and then we would delete the source physical share, along with its private driver storage18:41
ganso_in that new idea, it is certain that at some point, both backends will have a share with the same ID, like you said18:42
bswartzwhich driver would store in the private data storage?18:42
ganso_both, the private driver storage is stored per backend18:43
ganso_so, if backend A has the private storage entry "volume_id":"X" and backend B adds the private storage entry "volume_id":"Y", it does not overwrite, backend A will never find the value backend B added18:44
ganso_they can only read their own18:44
ganso_the host name is used as filter18:44
ganso_when retrieving the private driver storage18:44
bswartzis that how it's implemented now?18:45
ganso_if you think about it, it is quite a powerful thing, we can store any values we want that may assist in tracking the migration status in that private storage18:45
bswartzwith a host column in that table?18:45
ganso_as far as I understood, yes, unless I got it wrong18:45
bswartzyeah but the whole point of driver-private storage was that it's *for the driver only*18:45
ganso_yes18:46
ganso_I discussed that with vponomaryov18:46
bswartznow we're immediately overloading the feature so that 2 drivers can share it18:46
bswartzI don't like that18:46
ganso_I think it is quite wrong that the manager touches that18:46
ganso_not really sharing18:46
ganso_the manager would be handling all that stuff18:46
bswartzwell the manager shouldn't touch those fields18:46
bswartzif the manager needs more fields we can add more fields to the shares table or another table18:46
ganso_and I quite don't like the manager touching that as well... but he said it is okay... we may open it for discussion18:46
bswartzI don't like overloading that table with another purpose before any driver has even started to use it18:47
ganso_ /s/may/can18:47
bswartzthat's also an easy problem to solve though18:47
bswartzwe can add more columns to the share table or create a migrations table to work around those if needed18:48
ganso_I think this decision is essential for this approach18:48
bswartzI'm just trying to imagine what this would looks like18:48
ganso_if we are adding more columns I may even prefer to stick with 2 IDs18:48
*** fthiagogv has quit IRC18:54
*** zhonghua-lee has joined #openstack-manila19:00
*** zhonghua-lee has quit IRC19:16
*** MIDENN_ has quit IRC20:01
*** esker has quit IRC20:30
*** timcl has quit IRC20:33
*** akerr has quit IRC20:54
*** cknight has quit IRC20:57
*** mdenny has joined #openstack-manila21:17
*** lpetrut has quit IRC21:18
*** mtanino_ has quit IRC21:31
*** openstackgerrit has quit IRC21:36
*** openstackgerrit has joined #openstack-manila21:36
*** rcallawa has quit IRC21:46
*** rushil has quit IRC22:19
*** marcusvrn1 has quit IRC22:21
*** rcallawa has joined #openstack-manila22:34
*** rcallawa_ has joined #openstack-manila22:35
*** absubram has quit IRC22:37
*** rcallawa has quit IRC22:39
*** rcallawa__ has joined #openstack-manila22:39
*** rcallawa_ has quit IRC22:39
*** rcallawa__ is now known as rcallawa22:40
*** rushil has joined #openstack-manila22:41
*** akshai has quit IRC22:47
*** chlong has quit IRC22:52
*** ganso_ has quit IRC23:14
*** rushil has quit IRC23:20

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!