18:04:15 #startmeeting sahara 18:04:16 hi 18:04:16 Meeting started Thu Mar 13 18:04:15 2014 UTC and is due to finish in 60 minutes. The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:04:17 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:04:20 The meeting name has been set to 'sahara' 18:04:37 o/ 18:04:38 yeah, I've typed sahara, not savanna! 18:04:57 sav^hhara 18:05:11 #link https://wiki.openstack.org/wiki/Meetings/SaharaAgenda 18:05:30 #topic News / updates 18:05:33 folks, please 18:06:00 #info graduation review tc meeting logs http://eavesdrop.openstack.org/meetings/tc/2014/tc.2014-03-11-20.03.html 18:06:02 I think the client renaming is done 18:06:25 Mostly renaming activities for sahara-dashboard with occasional touchpoints on the other projects. Getting closer to done. 18:06:25 #info there are no concerns raised on tc meeting, waiting for voting 18:06:43 waiting waiting waiting 18:06:46 Good progress on AMBARI-1783. Will plan to incorporate into HDP plugin once GA. 18:07:02 mattf, yeah 18:07:02 actually I was 5 day off this week but started to rename main savanna service 18:07:05 SergeyLukjanov, voting next week? 18:07:28 aignatov, I can help you if you need it 18:07:36 ErikB2, are you removing puppet in that same effort? 18:07:37 ErikB2, technically, it's already started, but looks like tc folks waiting for review from ttx 18:07:42 The last TC meeting was pretty smooth (compare that to the incubation meeting, jeesh) 18:08:04 couldn't be smoother 18:08:10 tmckay: ok, we have a questions about backward compat 18:08:12 ErikB2, yup, the awosome progress 18:08:14 it was amazingly smooth 18:08:15 I've added support of multi-regions (on review, but for J) and added integration tests for IDH 3.0.2 (on review as well) 18:08:17 awesome* 18:08:17 mattf, not in the same motion, that will happen later (or not by us) 18:08:28 maybe they were tired from beating up on DinaBelova 18:09:08 or were preparing to neutron :) 18:09:15 aignatov, once heat becomes the default provisioning engine, will the Nova stuff still be there, or will that be removed? 18:09:19 mattf: no, it's because our excellent work since incubation 18:09:30 aignatov, ++ 18:09:32 aignatov, that certainly didn't hurt 18:10:00 ErikB2, depends on feature parity, but the plan is to remove it in J 18:10:32 any other news? 18:10:38 SergeyLukjanov, I would vote to not remove. 18:10:42 ErikB2: I'll do my best to apply nova for heat, actually there are some bugs/bps in heat which can help tightly integrate heat engine with both neutron-nova 18:11:05 ErikB2, why? 18:11:21 SergeyLukjanov, there are some distros out there that have chosen not to include heat. 18:11:22 removing isn't unreasonable, so long as we have feature parity 18:11:34 * mattf hmmmmms 18:11:58 guys, actually there are no huge efforts to be done to include nova with heat, really 18:11:59 it's integrated project, I think all distros will include it eventually 18:12:23 it's a big question for us for summit / Juno about removing/keeping direct engine 18:12:27 let's move on now 18:12:36 #topic Project naming status 18:12:43 SergeyLukjanov, I agree eventually that will happen, but it makes it impossible to use Savanna on anything w/out Heat. 18:13:10 #info all repos / lp projects / wiki spaces / teams / groups / mls / etc has been removed 18:13:31 ErikB2, this may be the first real upstream/downstream conflict. is the distro supporting sahara? 18:14:07 SergeyLukjanov: there are still 3 minor issues on the wiki, 2 on Sahara/How_To_Release, 1 on Sahara/BugTriage 18:14:13 #info gate is now blocked, I'm working on unblocking it 18:14:23 elmiko, it's on me, I remember 18:14:48 #info savanna-ci is now quite broken too (neutron issue) 18:14:59 I hope it'll be fixed tomorrow ^^ 18:15:16 currently we collected a lot of CRs for renaming 18:15:16 SergeyLukjanov: as for sahara-image-elements, it's been merged but we are waiting on a HortonWorks rename for the hadoop-swift image before we can eliminate all references to savanna 18:15:28 elmiko, great 18:16:12 I'll start merging some CRs when see gate working correctly and notify all team members about it 18:16:13 I'd prefer to merge all sahara main patches only WITH savanna-ci's +1 18:16:32 aignatov, ++ 18:16:37 aignatov, sure, we shouldn't brake own project :) 18:16:44 I am putting my trust in ci for thos 18:17:31 yesterday I sent patch for renaming swift-edp configs and now it fight with me: I can't understand - is it my patch introduce some bugs or intermittent issues with ci 18:17:54 aignatov, I'm afraid that neutron isn't working atm on savanna-ci 18:18:03 I'll check it after the meeting 18:18:16 dashboard tests are giving -1 now due to an old image in a testing lab 18:18:39 aignatov, so, how will hadoop itself be fixed? Doesn't the ".savanna" part of the url get handled in hadoop? Or am I wrong? 18:18:44 for swift, I mean 18:18:52 NikitaKonovalov, and so it was tested manually by aignatov and NikitaKonovalov 18:18:55 that will go away when nodepool rebuilds the snapshot 18:19:08 tmckay, it's about backward compat, next topic 18:19:17 any thoughts on renaming except backward compat? 18:19:38 tmckay: ".savanna" is handled in hadoop but just renaming to any word should work I hoped 18:19:57 I mean there is no hardcode ".savanna" in hadoop-swift.jar 18:20:14 only in openstack/savanna 18:20:19 aignatov, ah, okay. I wasn't sure how that happened. Nice. 18:20:19 sahara* 18:20:38 #topic Backward compatibility for renaming 18:20:45 we need to put bitcoins in a jar every time someone says savanna :) 18:20:52 lol 18:20:54 lol 18:21:02 :) 18:21:10 so, let's discuss backward compat 18:21:26 tmckay: the latest savanna-ci run show me that edp works at least in vanilla but in HDP it didn't go to SUCCEEDED :( 18:21:44 note: we already break it several times in Icehouse cycle (API, client) 18:21:45 and IDH went to Error...RRRRR 18:22:10 note: we introduced migrations in I, so, no way to migrate from 0.3 to Icehouse 18:22:23 aignatov, hmmm. do you want help debugging? 18:22:39 do we all agree that compat becomes of paramount importance once we've graduated? 18:22:44 note: it's a good practice to squash migrations for release to one migration "icehouse_release" 18:22:48 SergeyLukjanov: I'd prefer to forget about compatibility :) 18:22:54 I have my trusty laptop, which is easy to run integration tests on :) 18:23:01 mattf, ++ 18:23:14 mattf, ++ 18:23:22 mattf, agree 18:23:33 mattf, +1 18:23:36 tmckay: it would be great, especially EDP+not vanilla 18:23:39 yes 18:23:40 mattf, in case of graduation, Juno will be our first integrated release and so, we'll need to keep backward compat after it 18:23:47 if we decide compat isn't important w/ this rename then we should not even pretend -- we should not have a savannaclient at all, only saharaclient 18:24:10 mattf, what is the scope of breaking backward compat? 18:24:11 mattf, yup, just use them for transition and then remove 18:24:12 it's worse to provide a savannaclient that breaks api than it is to provide no attempt at compat 18:24:13 aignatov, okay. I've never successfully run one of the plugins locally on Fedora, but now is maybe the time to try (or get a RHEL box) 18:24:34 mattf, agreed 18:24:45 mattf, +1 18:24:51 mattf, ++, all or nothing, absolutely 18:24:53 elmiko, that's a good question. the api is an easy answer, but i don't think that's a complete answer. 18:25:15 mattf, i agree about providing a broken savannaclient worse than no compat 18:25:36 #startvote No backward compat for renaming? Yes, No, Abstain 18:25:37 Begin voting on: No backward compat for renaming? Valid vote options are Yes, No, Abstain. 18:25:38 Vote using '#vote OPTION'. Only your last vote counts. 18:25:42 to have it in logs 18:25:50 hold up 18:26:07 mattf, ? 18:26:08 what's the motivation for psuedo-compat savannaclient? 18:26:32 #vote yes 18:26:35 SergeyLukjanov, you said somethng about "transition" - what's the motivation and scope for that transition 18:26:35 elmiko, some existing objects in the savanna db might fail after upgrade 18:26:35 elmiko, strings that contain savanna (bitcoin) 18:26:37 #vote yes 18:26:54 mattf, only to go through the gate 18:27:00 am, I thought we are not going to keep savannaclient 18:27:02 only saharaclient 18:27:24 tmckay, wouldn't we want to catch those and upgrade them asap? 18:27:27 SergeyLukjanov: so once gate issues resolved, we will drop it, right? 18:27:40 savannaclient < 0.5 should exist 18:27:41 mattf, because of circular dependency between sahara / sahara-client / devstack / devstack-gate / tempest 18:27:41 SergeyLukjanov, so we either have a single commit we force through the gate or we have a commit that passes the gate and a second commit that removes the compat? 18:27:55 mattf, the second option 18:27:58 dmitryme, yu[ 18:28:02 elmiko, well, to catch them means an alembic database migration script. To have 0 backward compat means, they fail. 18:28:04 #vote abstain 18:28:07 Make a new database 18:28:14 SergeyLukjanov, those are the 2 options though? 18:28:16 tmckay, got it 18:28:18 mattf, it'll make us able to always pass CI 18:28:35 mattf, I mean that we'll "we have a commit that passes the gate and a second commit that removes the compat" 18:28:39 #vote yes 18:28:47 #vote yes 18:29:06 #vote yes 18:29:15 #vote abstain 18:29:19 related to this is if we should just scrap all db migrations, after renaming just have folks recreate their db --- no compat 18:29:26 #vote abstain 18:29:34 * mattf feels very railroaded 18:29:36 mattf, that is a good question. 18:29:55 it's not clear to me that we understand the scope of what we're voting on 18:29:55 does a no kill the vote? 18:30:18 mattf, i know i don't... 18:30:24 I agree that perhaps I'm just not clear enough to make a yes vote out of me. 18:30:32 we're voting to not keep backward compat, we could discuss details separately 18:30:35 such as migrations 18:30:53 I can close it and we'll vote after discussing details 18:30:59 I think I would at least endorse migrations. It's painless. 18:31:15 if we vote to do no compat that means we don't provide migrations 18:31:22 SergeyLukjanov: i like the idea of killing backward compat, but i really don't know the scope of that decision well enough 18:31:28 tmckay, I don't like data manipulations inside the migrations 18:31:36 we can't do a blanket vote and then piecemeal details. 18:31:39 having to dump a database because of code change is really, really bad form. I used to make users do it in another project, until I fixed it :) 18:31:40 cant/shouldnt 18:31:49 tmckay, we'll need to have a good tests for it 18:32:02 mattf, it's about overall approach 18:32:11 SergeyLukjanov, ack 18:32:13 #closevote 18:32:18 #endvote 18:32:19 Voted on "No backward compat for renaming?" Results are 18:32:20 Yes (5): ruhe, alazarev, dmitryme, SergeyLukjanov, sreshetnyak 18:32:21 Abstain (3): NikitaKonovalov, elmiko, crobertsrh 18:32:30 #undo 18:32:32 Removing item from minutes: 18:32:35 #undo 18:32:36 Removing item from minutes: 18:32:52 #topic Backward compatibility for renaming 18:32:53 what are the specific areas where compat is in question? 18:33:03 API 18:33:10 savanna-db 18:33:12 we have client, we have db, ui urls 18:33:13 EDP database objects 18:33:14 #1 migrations for .savanna suffix and savanna-db prefix 18:33:15 savanna_url 18:33:30 ErikB2, API isn't changed 18:33:33 savanna_use_neutron etc...config 18:33:46 SergeyLukjanov, property names will 18:33:47 #2 configs like SAVANNA_USE_NEUTRON in dashboard 18:33:51 Also Swift 18:34:00 .savanna in Swift, yes 18:34:08 ErikB2, aignatov, it's #1 that I mentioned 18:34:18 SergeyLukjanov, yez 18:34:24 but I don't know how to test all it today 18:34:48 The "savanna_url" param in the Client.__init__() method. 18:34:51 personally, I think that it's useless to keep #2 old-named configs 18:35:07 SergeyLukjanov, let's build the list first 18:35:25 tmckay, we've agreed that there is no need to keep savannaclient, so, there is no need to keep savanna_url in saharaclient 18:35:25 what i'd give for a notepad 18:35:34 SergeyLukjanov, I agree. I would like to see us move forward with as little baggage as possible 18:35:37 SergeyLukjanov, ack 18:35:39 * SergeyLukjanov creating an etherpad to list options 18:36:00 https://etherpad.openstack.org/p/savanna-renaming-backward-compat 18:36:38 API (payloads), savanna-db, EDP db obks, client (modules and api), config (e.g. SAVANNA_URL etc) 18:36:40 guys, the most important thins that there is no much time to do good backward-compat code since release is coming 18:36:45 API (payloads), savanna-db, EDP db objs, client (modules and api), config (e.g. SAVANNA_URL etc) 18:37:22 aignatov, agreed, and additionally, it sounds useless due to the fact that Icehouse is our first aligned release 18:37:26 API (payloads), savanna-db, EDP db objs, client (modules and api), config (e.g. SAVANNA_URL etc), swift integration 18:37:30 aignatov: I think people want to have a clear view of what we will miss if we drop compatibility 18:37:33 anything missing from ^^? 18:37:38 mattf, please, add to the etherpdad 18:37:57 sure 18:38:08 mattf: docs with cleat description 18:38:12 *clear 18:39:03 I mean if we'll keep compat we should add notes to all compat points, we need to describe that here you can use savanna_url and sahara_url as well 18:39:15 just for example 18:39:19 it's time 18:39:36 aignatov, and we'll need to remove this compat anyway 18:39:52 to continue improving project 18:41:02 elmiko, regarding the savanna-image-elements change you mentioned earlier, can you be more specific about "we are waiting on a HortonWorks rename for the hadoop-swift image before we can eliminate all references to savanna" ? What exactly needs to be changed? 18:41:05 mattf, what's the REST API payloads? 18:41:15 SergeyLukjanov, the json prop names ErikB2 mentioned 18:41:23 e.g. tags 18:41:45 mattf, Image Regitry 18:41:52 I'll add a separated point for it 18:42:08 k 18:42:19 bob_nettleton: in the file elements/hadoop-hdp/source-repository-hadoopswift, the link to the hortonworks s3 contains a savanna dir name. that's it 18:42:40 elmiko, ok, thanks for clarifying this. 18:42:47 bob_nettleton: np :) 18:43:21 so what would be our final decision about compatibility? 18:43:42 configuration, python client, rest api, all database storage, documentation 18:43:47 ^^ my high level summary 18:44:07 documentation seems strange in this list 18:44:08 a decision for no compat will essentially mean you cannot upgrade across the rename 18:44:12 you'll have to create your db 18:44:18 update all your scripts to use new names 18:44:29 handle any diff in rest calls 18:44:33 use new configuration 18:44:41 SergeyLukjanov, yeah, but it's true 18:44:49 am i missing anything? 18:44:54 the latest released savanna version is 0.3 and you already need to do all stuff mattf mentions 18:45:25 SergeyLukjanov, yup, but now we're part of openstack. we should at least go into the decision with all the relevant information. 18:45:39 i'm not advocating either way atm. i just want us to make an informed decision 18:45:49 mattf, sure 18:45:59 mattf, I'm not against this discussion 18:46:00 let's give it another minute to see if anyone thinks of something missing from the list 18:46:20 SergeyLukjanov, mattf, what is the downside to carrying backward compat into Icehouse and the removing before the next major release? 18:46:46 elmiko, it's not clear that we currently have a mandate to maintain compat and it's a lot of work 18:46:50 mattf: but I don't see other options, adding compat will take a lot of efforts 18:47:00 elmiko, we'll need weeks of coding and testing to make it 18:47:08 one missed point is new tests, I mean if we want to keep compat we need to write more tests around that 18:47:10 got it, thanks 18:47:19 ok, sounds like we agree that the list is complete enough 18:47:28 and it's not clear for me what's the starting point for such backward compat - 0.3 or mid Icehouse 18:48:04 mattf, + urls in UI probably is a bit important too 18:48:06 ErikB2, no compat may have the biggest impact on you. what's your opinion? 18:48:25 SergeyLukjanov: it seems that 0.3 is already lost :) 18:48:49 mattf, perhaps, but my vote would be for no compat until start of juno 18:49:03 I remember one my patch that breaks.... hmm...ok... I will silent 18:49:34 ErikB2, ok 18:49:38 aignatov: it breaks even working 0.3 now :) 18:49:38 ErikB2, ++ 18:49:50 and it's been raised a number of times that compat is very expensive right now 18:50:01 alazarev: psst, silence 18:50:08 xD 18:50:09 SergeyLukjanov, and we agree that our mandate for compat really starts after integration? 18:50:13 the backward compat is veey important starting from Juno release, especially API compat 18:50:29 mattf, first integrated release I think 18:50:30 mattf: ++ 18:50:34 +1 on starting compat from I release 18:50:55 anyone have any questions about what we're voting on when we say no compat? 18:51:07 +1 on start compat from Icehouse, yes 18:51:09 Nope, I feel better about voting now 18:51:13 there is a diff between starting from I release vs. starting from J (first integrated release) 18:51:27 mattf, we vote "yes" for no compat? 18:51:43 so, any thoughts on I vs. J? 18:51:47 so I vote "yes" for no compat as well 18:51:47 SergeyLukjanov: you mean I release, or I start? 18:51:49 elmiko, i'm just curious if there are any more questions before we vote 18:52:10 mattf, none from me, i feel better about the scope now 18:52:11 SergeyLukjanov: or mid 18:52:14 ok, let's re-word 18:52:37 would we like to have migrations and other compat stuff from Icehouse tag to Juno tag 18:52:38 "icehouse can break everything" --- reworded :) 18:52:59 or starting from Juno tag (potentially first integrated release) 18:53:01 SergeyLukjanov: what do you mean by integration release? 18:53:23 integrated release is the first release being the integrated project 18:53:31 with sync gate 18:53:38 and etc. 18:53:53 let's define compat-start point, everything before it is not upgradable 18:53:55 that means integrated into full openstack? 18:54:19 elmiko, it means that we'll graduate from incubation and release 18:54:33 SergeyLukjanov: ok, understood ty 18:54:34 alazarev, +1 18:54:36 elmiko, Juno is the first release when we could be part of 18:54:41 will we vote today again? 18:54:47 aignatov, yep 18:54:51 5 mins left 18:54:59 * elmiko still getting up to speed on terminology 18:55:08 elmiko, it's a little funky 18:55:10 SergeyLukjanov: do you mean j-1? 18:55:16 alazarev, nope 18:55:24 dev releases are dev releases :) 18:55:58 SergeyLukjanov: sahara release at time of I release (non-integrated)? 18:56:04 lets move this topic to some design session and discuss all rules about compatibility 18:56:09 let's vote for not keeping backward compat for renaming 18:56:16 ack 18:56:22 and than decide the start point for keeping full backwark compat 18:56:49 #startvote Should we keep backward compat for renaming? Yes, No, Abstain 18:56:49 Begin voting on: Should we keep backward compat for renaming? Valid vote options are Yes, No, Abstain. 18:56:50 Vote using '#vote OPTION'. Only your last vote counts. 18:56:52 aignatov, alazarev, it may be easier to just define a date when we start maintaining compat 18:57:15 #vote no 18:57:16 #vote no 18:57:16 #voite no 18:57:20 #vote no 18:57:20 #vote no 18:57:25 #vote no 18:57:26 #vote no 18:57:34 #vote no 18:57:35 #vote no 18:57:49 #vote no 18:57:56 #vote no 18:58:01 *expecting mattf to vote for himself* 18:58:13 ErikB2, that's so last week! 18:58:17 I know 18:58:19 #vore no 18:58:26 #vote no 18:58:26 #vote no 18:58:34 #vote yes 18:58:42 just for some drama :) 18:58:46 lol 18:58:48 :) 18:58:53 aignatov, i think we already filled the drama quota 18:59:07 New vote...aignatov handles all pre-renaming backward compatability changes. 18:59:10 30 secs more 18:59:12 mattf: lol, exactly 18:59:14 vote yes 18:59:18 crobertsrh ++ 18:59:23 crobertsrh: +1 18:59:28 no compat means I have mess with one of my patches again, sigh 18:59:32 #endvote 18:59:33 Voted on "Should we keep backward compat for renaming?" Results are 18:59:35 Yes (1): aignatov 18:59:35 tmckay, yeah 18:59:36 No (11): bob_nettleton, NikitaKonovalov, dmitryme, elmiko, ErikB2, crobertsrh, tmckay, mattf, SergeyLukjanov, alazarev, sreshetnyak 18:59:55 crobertsrh: lol! 19:00:04 so, let's start next meeting from discussion about the starting point for full backward compat 19:00:10 than you all folks 19:00:12 #endmeeting