20:01:40 #startmeeting trove 20:01:40 Meeting started Wed Sep 18 20:01:40 2013 UTC and is due to finish in 60 minutes. The chair is SlickNik. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:01:41 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:01:41 \o 20:01:44 The meeting name has been set to 'trove' 20:01:53 #link https://wiki.openstack.org/wiki/Meetings/TroveMeeting 20:01:54 #link https://wiki.openstack.org/wiki/Meetings/TroveMeeting#Agenda_for_the_next_meeting 20:02:05 why no #disagree i wonder 20:02:05 0/ 20:02:12 SlickNik: plz refresh your meeting agenda i just finished updating. 20:02:12 \..o../ 20:02:12 Let's get started with with the Action Items 20:02:19 o^/ 20:02:26 cp16net: Bicep flexing? 20:02:29 o/ 20:02:34 always 20:02:35 lol 20:02:35 cp16net: just did, thanks for heads up 20:02:39 np 20:02:46 #topic Action Items 20:02:56 #link http://eavesdrop.openstack.org/meetings/trove/2013/trove.2013-09-11-20.00.html 20:03:04 o/ 20:03:20 1. cp16net add the db model to schedule_task 20:03:23 https://wiki.openstack.org/wiki/Trove/scheduled-tasks#Scheduled_Task_Schema 20:03:26 #link https://wiki.openstack.org/wiki/Trove/scheduled-tasks#Scheduled_Task_Schema 20:03:35 thanks cp16net 20:03:58 cp16net: what about frequency ? 20:04:06 thats at least what was asked for 20:04:12 did we came into suggestions ? 20:04:18 dmakogon_: :-/ 20:04:21 doh 20:04:26 i hanvt updated it with that 20:04:31 lol 20:04:39 so, i'll tell this part 20:04:39 i'll update more on this when the main topic comes up 20:04:50 cp16net / dmakogon_: Let's review what's up and discuss it later 20:04:57 current schema missing "frequency" field 20:04:58 sounds good 20:05:05 ok 20:05:07 That's all we have with Actions items. 20:05:11 yay 20:05:17 yeap 20:05:19 cp16net: what is deleted vs deleted_at? 20:05:28 time and flag 20:05:33 #topic Launchpad blueprints can only be edited by core members, sooper inconvenient (esp) 20:05:37 they are both datetime 20:05:38 kevinconway: is it deleted bool and time it was deleted 20:05:54 doh... copy pasta 20:05:56 cp16net: lol, another miss)))) 20:06:08 cp16net: why have both a date and flag? 20:06:17 cuz it's the openstack way 20:06:20 would delete NOT NULL indicate not deleted? 20:06:20 indexing maybe? 20:06:21 ok refesh :) 20:06:22 kevinconway: unique constraint problems 20:06:46 amcrn: what unique constraints? 20:07:12 kevinconway: easier to build a boolean into an index than a datetime. 20:07:18 Is this really the case? (i.e. only core members have permissions on blueprints?) 20:07:31 That seems like something we definitely need to fix. 20:07:42 SlickNikL why do we need this ? 20:07:43 Well I'm on -drivers, let me try to fiddle with a BP 20:07:49 yeah where is esp? 20:08:04 he had the issue. 20:08:05 not at his desk 20:08:25 but I'm going to action it to figure it out for myself. 20:08:31 So that we can fix it if it's an issue. 20:08:41 sounds good 20:08:59 #action SlickNik to check with hub_cap to make sure all contributors can create/edit blueprints. 20:08:59 SkickNik: i think that this is no some kind of issue, or am i wrong ? 20:09:03 It would appear all I can mess with is the Goal of the few BPs that I looked at just now 20:09:03 maybe it was because they didnt create it 20:09:23 I can confirm I've created blueprints, but there are a few fields that are not editable 20:09:33 I can't edit ones not assigned to me 20:09:43 amcrn: which fields ? 20:10:01 dmakogon_: don't remember off-hand, I can re-check later. 20:10:11 on a side note... is reddwarf-drivers matter? 20:10:11 amcrn: ok 20:10:15 amcrn: okay, I'll follow up on this. I think that every contributor should be able to create & edit bps (regardless of who initially created the bp) 20:10:20 it maybe related tho 20:10:37 cp16net: I believe reddwarf-drivers only matters for access to rdjenkins 20:10:39 reddwarf-drivers vs trove-drivers 20:10:43 ok 20:10:54 i wasnt sure 20:11:07 Let's move on, I'll follow up with hub-cap to get this sorted out. 20:11:14 SlickNik: cp16net: could we extend -drivers teams ? 20:11:29 #topic databases/users validation or api change 20:11:48 dmakogon_: What we really need is a -contributors team. 20:11:55 for this topic, review should be covered by tests 20:12:00 fwiw i don't necessarily think it's wrong for our api to allow grants to ghost dbs 20:12:08 SlickNik: cp16net: yes, good point 20:12:11 datsun180b: Me neither. 20:12:17 datsun180b: will that work on postgres? 20:12:33 I asked around Rax because I figured this would be viewed as a backwards compatibility issue, but no one has objected. 20:12:33 or sql-server, or oracle, or etc. 20:12:36 why would allow that, seems sane to require the db to exist 20:12:45 amcrn: why don't we deal with those as they're implemented 20:12:49 So I'll remove my minus 2. 20:13:15 vipul: I agree the validation makes sense, the issue was people could have written scripts or code that uses the Trove API and creates databases with users that have access to ghost databases 20:13:29 So general backwards compatibility stuff 20:13:45 grapex: i see.. 20:13:52 who are we to restrict our users' access to mysql in this case. surely trying to grant permissions in a less laissez-faire impl would cause a problem, so we'd need to intercept that error 20:14:12 agreed 20:14:13 then why do we valid the hostname? 20:14:18 However I can't find people who object, so I'm ok. At heart I'd prefer the API make sense than be backwards compatible. Although in this case it *does* kind of match MySQL semantics... 20:14:35 validate* 20:14:48 hub_cap made the point to me though that for our MySQL agnostic API we shouldn't use MySQL semantics as a guide. 20:14:48 that's a fair question 20:15:28 #agreed 20:15:29 SlickNik: cp16net: vipul: datsun180b: should validation should be custom for each database engine/type ? 20:15:41 the precedent has been to validate input thus far, so I don't believe that validating properties in the same request against each other violates that 20:16:09 dmakogon_: i think we're trying to avoid that 20:16:12 the intention is clear from the user, to associate the user to a database in a create request, if the database isn't in the global list, i'd argue the user clearly doesn't understand what they're requesting for 20:16:14 avoid it a lot 20:16:17 oh for the uniqueness problem i totally agree 20:16:55 SlickNik: cp16net: vipul: so, what the conclusion ? 20:17:01 *what's 20:17:15 i guess the other way to interpret your example request would be to create the missing databases as they're needed but that's likely overstepping our bounds there, being too assuming 20:17:29 datsun180b: I'd agree 20:17:36 kevinconway: one word for you... "users" 20:17:41 shhhh 20:17:45 imsplitbit: SHHH 20:17:46 datsun180b: Honestly I think it's kind of a pain we have to specify them twice... 20:17:46 I think our api, regardless of the mysql impl makes sense to validate given that we validate other things 20:18:11 isn't the intent to remove users/databases from the create request anyway? 20:18:19 SlickNik: cp16net: vipul: imsplitbit: i think we should do double validation 20:18:20 honestly i don't know why users and databases are options in create-instance 20:18:37 maybe that's the problem here, we have ways to create databases, users, and manage permissions 20:18:41 SlickNik: cp16net: vipul: imsplitbit: at client and at API 20:18:54 ah, the ol' double hull solution 20:18:56 datsun180b: It makes sense- that way you don't have to poll the API, waiting for it to provision before you add a user and database your app needs 20:19:09 because we'll never need three! 20:19:13 grapex +1 20:19:36 datsun180b: I think that's possibly a popular use case 20:19:42 well stone me to death if you will but that sounds like a job for a workflow 20:19:53 why not let them create a security group or configuration group in create also then? 20:20:03 I know if we took it out the Rackspace control panel would end up recreating it, and as a former UI guy I feel like in those cases why not have it in the API? 20:20:04 they 20:20:42 okay, I think we may need to move on soon in the interest of time. 20:20:53 anyway, I think we're diverging a bit. Whether the database/users stuff gets ripped out in the future or not, should we validate what we currently have? 20:21:11 Yes 20:21:24 But what I'm hearing is that validation is a good thing in this case. 20:21:29 i can't build an argument strong enough to say ghost grants are okay 20:21:31 SlickNik: I think we're all ok with the validation. 20:21:43 And given that grapex is okay with the backward compat issue, amcrn can proceed. 20:21:44 grapex: yes 20:21:51 it's hard to argue against validation 20:22:04 +1 20:22:21 kevinconway: I disagree- I think you can easily build a valid argument against it. 20:22:22 sounds like we have consensus.. 20:22:27 next topic 20:22:28 Let's move on to the next topic. 20:22:30 grapex: har har 20:22:47 * grapex hears the sound of an audience laugh track somewhere 20:22:49 #topic Cluster API. Provisioning part specs (dmakogon) 20:22:53 next topic is mine 20:23:01 SlickNik: cp16net: vipul: imsplitbit: grapex: https://gist.github.com/crazymac/6580664 - take a look iat this gist 20:23:14 i need some comments 20:23:15 i feel left out now 20:23:32 you're not alone kevinconway i'm on the sidelines with you 20:23:51 lol, if you're not working at rax, you don't get included in dmakogon_'s nick-list 20:24:04 lolz 20:24:06 :p i don't work for rax 20:24:07 lol 20:24:07 lol 20:24:16 vipul: you're special ;) 20:24:27 independent, even :P 20:24:28 thought so 20:24:43 dmakogon_: what's the difference between that gist and the Clustering API wiki written by imsplitbit? Is the idea that it's defining the workflow or ? 20:24:45 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: so, what do you think ? 20:24:54 what amcrn said 20:24:58 what are we looking at.. 20:25:03 dmakogon_: is this pseudo code? 20:25:03 and how is this different 20:25:08 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: my gist is about implementing, not writing new spec 20:25:15 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: yes 20:25:26 most of the create is implemented 20:25:30 as is delete 20:25:37 why am i still not on the nick list? 20:25:39 theres some permissions checking that needs to be done for delete 20:25:43 lol @ kevinconway 20:25:46 lol @ kevinconway 20:25:48 kevinconway: cause you just want to talk about users 20:26:00 I also work at rax and am not included =( 20:26:09 kevinconway: Change your nick to "user-master" 20:26:15 i thinks we should move instance group provisioning to heat 20:26:33 * redthrux THERE ARE NO EMOTIONS IN OPENSTACK MEETINGS 20:26:41 dmakogon_: dropping the H bomb over there 20:26:44 * redthrux PUT THEM ON THE SHELF WITH YOUR EGO 20:26:44 creating and joining instances into one network although should be moved into heat 20:27:02 :D 20:27:09 dmakogon_ i guess it looks reasonable.. although impl is usually best described in code :) 20:27:33 the current api stuffs metadata into a new instance and creates it 20:27:43 hub_cap has done some work already to integrate HEAT into instance provisioning in some way 20:27:46 i'm pretty sure it merged 20:27:48 I don't know that heat is the right or wrong tool for that yet 20:27:52 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: firstly, i decided to write sone pseudo-code spec for implementing provisioning part 20:28:18 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: hub_cap had done only single instance provisioning 20:28:21 dmakogon_: So you should separately touch base with imsplitbit, since he is saying the creat eis mostly done 20:28:22 but if instance provisioning gets done in heat then cluster will inherit it as it creates instances using the instance model 20:28:51 +1 vipul 20:29:16 I'd hate for us to have two factions working on competing versions of the _same_ thing. 20:29:20 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: i'm already touched with him, and i pending approve to do some code for cluster API provisioning 20:29:22 the basic gist of cluster.create currently just looks at the number of specified instances and calls Instance.create(blah) x number of times where x is the number of instances requested to be created for the cluster 20:29:23 afaik, cluster provisioning was supposed to be solely through Heat.. but need to confirm with hub_cap 20:29:34 yeah, we should work together to produce two versions of the same thing 20:29:38 lol 20:29:46 heh 20:29:50 :) 20:30:09 dmakogon_: Are you looking for any other feedback? 20:30:23 imsplitbit + dmakogon_: Can we see an update to the blueprint/wiki on how you'll handle pdmars Configuration Groups as well as other blueprint considerations? 20:30:26 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: i'm not working on the same thing as imsplitbit worked on 20:30:30 dmakogon_: I am at the linux plumbers conf but we can chat more about this on Monday 20:30:45 and of course today here 20:30:57 imsplitbit: ok 20:31:06 Okay, let's move on to the next topic. 20:31:11 #topic Rollback on failre. ForceDelete (dmakogon) 20:31:20 https://gist.github.com/crazymac/6613436 20:31:23 #link https://gist.github.com/crazymac/6613436 20:31:31 waiting for comments 20:32:09 dmakogon_: So it seems like the gist is we need to refactor task manager to be a bit smarter. 20:32:22 I question automatically deleting instances 20:32:35 vipul: that makes me a bit nervous 20:32:37 why not extend the delete instance API? 20:32:44 accept a --force = true 20:32:45 or whatever 20:32:54 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: that is what i'm asking about 20:32:55 which ignores the state teh Guest is in, and deletes 20:33:05 vipul: +1 20:33:10 i'm not a fan of automatically deleting instances either, but when this was first brought up, all I could do was fight for the ability to turn it off 20:33:19 but we need rollback for each component on forceDelete 20:33:24 amcrn lol 20:33:29 vipul: We have a reset task status method that can help a bit in these cases. 20:33:32 I'm good with adding a —force flag to delete as well. 20:33:40 grapex: that's only for admins .. 20:33:48 grapex: reset task status is for mgmt-client 20:33:51 vipul: I think datsun180b did work to make that method also delete pending backups for screwed up instances. 20:33:56 SlickNik vipul: Good point 20:34:06 i guess you could have 'force delete' invoke that 20:34:21 why can't we just honor all deletes without the force? 20:34:22 I think the original motivation was that if the instance was that borked, an admin should figure it out and report a bug, and over time all bugs would dissappear and everything would be magical! 20:34:26 :D 20:34:29 right, if your instance gets stuck and is unresponsive, shouldn't you be calling support anyway 20:34:37 +1 datsun180b 20:34:55 if i recall the mgmt client delete is a little more powerful than the normal user delete 20:34:56 that's fair.. but for those that don't... 20:35:02 makes sense to just introduce a --force 20:35:04 i could see a case for instance delete --really --imeanit --confirm 20:35:06 robertmyers: In some cases a delete on a resource that is in some state can lead to other bugs 20:35:14 but i dont recall the in's and outs of it 20:35:23 * grapex runs through the halls of his mental labyrinth trying to remember an example of this. 20:35:29 grapex: i believe the rabbitmq connection is one of those things 20:35:40 guest would not disconnect? not delete queue? 20:35:40 i thought if the nova instance was in an active state it would delete even if the instance was in "building" from trove 20:35:49 maybe we need to loosen up the allowed states? 20:35:53 For example, let's say an instance is simply taking forever, so a delete goes through 20:35:55 and i imagine there's some work to decouple/deallocate potential storage for example 20:35:56 but the state is BUILDING 20:36:00 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: so, i'm still working on rollbacks, than i'll move all methods into one API call 20:36:03 robertmyers: puerto rico is a good start 20:36:14 kevinconway: ha 20:36:20 Then it gets deleted, but as it does, the thread which sends the billing event fires. 20:36:53 so you get a delete before a create? 20:37:03 vipul: Yes 20:37:05 aka free money 20:37:06 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: we should check statuses of each component to be ACTIVE/COMPLETED/FAILED to be deleted 20:37:26 I think at some point we had evidence such a thing was happening (in staging thankfully) 20:37:29 This was a while back 20:37:52 I think the building state is a little too vague 20:38:01 maybe more states 20:38:08 vipul: I think the problem though is that a delete call would need to cancel threads of execution in flight 20:38:14 anectodal evidence is the strongest kind, of course 20:38:17 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: so, did i get approve for workflow of forceDelete ? 20:38:39 dmakogon_: we're still discussing it. 20:38:51 possibly need to table the discussion for after the meeting. 20:39:00 Since we still have other topics to cover. 20:39:04 sounds to me like we ned to talk it over more 20:39:15 i would favor forceDelete over auto-delete any day though 20:39:21 +1 20:39:23 chat later .. move on 20:39:25 Let's move this discussion to #openstack-trove after the meeting. 20:39:28 vipul: We used to have a Reaper... 20:39:29 i think if you're going to jettison the warp core of an instance, it's generally a good idea to check in with Scotty first 20:39:47 grapex: I think the reapear makes sense.. for stuck in deleting though 20:39:47 #topic Anoying fails in reddwarf-jenkins due to memory exceeded exceptions (dmakogon) 20:39:49 well i want to mention that there are instances where we need to remove the nova-bones when RPC times out or something - 20:39:50 not for stuck in build.. 20:39:52 crap 20:39:56 I MISSED IT 20:40:01 * redthrux depressed 20:40:17 * redthrux slams closet door 20:40:20 redthrux: we still read it. 20:40:31 :D 20:40:31 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: how could we extend resources of HP Cloud ? 20:40:35 kidding here 20:40:47 dmakogon_: You can buy us some more servers? 20:40:54 lol 20:40:56 Send money via PayPal 20:40:59 so I can get a cut 20:41:00 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: nope 20:41:07 ;) 20:41:20 but I did want to mention i see instances all the time that have disrupted RPC or simply time out on a volume task 20:41:30 So every so often we have instances that are orphaned. 20:41:51 It's an artifact of how the gerrit trigger plugin interacts with jenkins. 20:41:52 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: yes 20:41:56 and while we're at it SlickNik can you make my HP cloud account unlimited servers for free? 20:42:04 I cleaned them out a couple of days ago. 20:42:08 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: lol 20:42:25 so it's worthwhile to mention we need to actually have the reset_task_status actually do what it says *regardless* 20:42:38 but it happening again and again ==(( 20:42:39 so then we can issue a delete and have trove dispose of the nova body 20:43:01 The only really good answer I have for now is if we're in this state find some HP trover to clean out the orphaned instance. 20:43:09 and perhaps wear nova's uniform to continue its mission 20:43:11 redthrux++ 20:43:12 But we should be retiring rdjenkins soon. 20:43:20 So it should be only temporary. 20:43:25 redthrux: the reset-task-status is useless.. still cannot delete after that 20:43:38 SlickNik: cp16net: vipul: imsplitbit: amcrn: redthrux: kevinconway: dmakogon_: grapex: what do we get after jenkins? 20:43:40 aw that's a shame to hear 20:43:49 i think it's not resetting service_status? 20:43:49 datsun180b: Please move this conversation to #openstack-startrek-metaphors 20:43:51 ok 20:43:56 lol @ kevinconway's nick list. 20:44:25 kevinconway: we should be moving to tempest soon, so we'll have the openstack-ci team's resources. 20:44:53 cool.. i don't know how many more 16GB 8 core vms HP will give us 20:44:55 btw, what about Tempest tests ? 20:45:24 what is current status of Tempest tests ? 20:45:28 dmakogon_: hub_cap had started on that. You might want to ping him about it when he's around. 20:45:31 dmakogon_: As I understand it hub_cap is looking into it. 20:45:41 let's move on to the next topic 20:45:46 #topic my.cnf Configurations changes (cp16net) 20:45:47 thanks 20:45:47 #link https://gist.github.com/cp16net/08761477cf9ce7f5c79b 20:46:13 so i made some comments on this after looking at amcrn and pdmars 20:46:58 nice, thanks cp16net 20:47:13 why do you apply Configurations only at mysql ? 20:47:14 we talked about adding a POST for the complete replacement of the parameters set 20:47:21 and a PUT for individual ones 20:47:25 Good stuff cp16net. thanks! 20:47:26 I agree w/ the PUT/POST concept 20:47:33 #agreed 20:47:46 looks good cp16net 20:48:05 I don't see a comment on the last bullet though, any ideas? 20:48:06 sounds great use of http verbs 20:48:10 that was one of the biggest changes that affects the api 20:48:19 has that stuff been merged yet? 20:48:46 about implementation, we need different processors of Configuration 20:48:50 i have not seen anything about the service+version 20:49:01 just talks about it 20:49:11 cp16net: right, so i say we wait until that's there and then work to support it 20:49:12 because different database engines have different config types 20:49:18 no implementation to speak of 20:49:21 key=value, YAML 20:49:24 cp16net + pdmars: one sec, linking forthcoming 20:49:32 i'm talking about future 20:49:37 why not a jsonschema? 20:49:37 that could be added in the validation rules 20:49:47 dmakogon_: are you saying YAML as API input? 20:50:05 kevinconway no 20:50:38 that can be changed when it comes 20:50:46 explain more of the key/value, yaml. I don't understand. 20:50:54 Hai 20:50:56 cp16net + pdmars: https://gist.github.com/amcrn/b3d35de76096dff2839a 20:50:57 kevinconway: about converting Configuration object into key=value/YAML configuration file 20:51:01 it's a rough draft, wrote it up last night 20:51:05 hey hub_cap 20:51:10 time check. 20:51:15 hub_cap hi 20:51:19 you mean as API output? 20:51:28 oh this was from what hub_cap talked about 20:51:32 I'm early. For the next meeting!! 20:51:43 kevinconway: what do you mean "API output" ? 20:51:47 Oh condemn me cp16net 20:51:49 lol 20:51:51 amcrn: thats alot to read right now 20:51:58 I don't even know what we are taking about LOL 20:52:01 hub_cap: once again traveling on Wednesday... 20:52:03 let's move on to the next topic. 20:52:08 Special move 20:52:09 dmakogon_: you are suggesting YAML as the return from an API call? 20:52:15 dmakogon_ / kevinconway please discuss offline. 20:52:26 #topic Scheduled task wiki update (cp16net) 20:52:31 SlickNik: but we're so far away. phone bill will be super expensive. 20:52:38 Skype 20:52:42 ;) 20:52:53 side topic: my ios 7 update finally went through after the 10th try! 20:52:54 passenger pigeon :) 20:52:55 dmakogon_: expect a letter from me in a few weeks 20:53:01 #link https://wiki.openstack.org/wiki/Trove/scheduled-tasks#Scheduled_Task_Schema 20:53:03 telegram? 20:53:08 kevinconway: ok 20:53:16 SlickNik: Carrier. 20:53:16 vipul: nice. how is it? 20:53:29 esp: installing 20:53:30 #link https://github.com/cp16net/trove/commit/c90b77fd0441e91ea4129f598718122dff1eb6c0 20:53:42 so this is the changes i've made toward the scheduled task 20:53:42 hub_cap: passenger, since it's extinct ;) 20:54:06 vipul: k, I want a live demo! 20:55:05 anything to discuss ? 20:55:09 cool cp16net great start 20:55:21 cp16net: will look at it soon and send you some feedback 20:55:30 yeah i've worked out the inital api 20:55:32 looks good at a cursory glance. 20:55:59 that will be the WIP branch 20:56:20 sounds good. 20:56:29 Okay, let's move on. 20:56:33 sounds good 20:56:34