18:08:02 #startmeeting sahara 18:08:03 Meeting started Thu Mar 27 18:08:02 2014 UTC and is due to finish in 60 minutes. The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:08:04 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:08:06 The meeting name has been set to 'sahara' 18:08:20 hello SergeyLukjanov 18:08:52 hello 18:08:57 * SergeyLukjanov completely forgot about meeting 18:09:33 #link https://wiki.openstack.org/wiki/Meetings/SavannaAgenda 18:09:34 ookay 18:09:47 #topic Project renaming status 18:10:17 so, I think that we could say that renaming successfully completed except several very nit places 18:10:30 * SergeyLukjanov going to update wiki page about meetings after the meeting 18:10:40 #info enaming successfully completed except several very nit places 18:11:04 #undo 18:11:05 Removing item from minutes: 18:11:11 #info renaming successfully completed except several very nit places 18:11:22 any more thoughts on it? 18:11:28 are we going to rename savanna-ci? 18:11:46 alazarev, yup, after rc1 I think to not break dev process 18:12:07 oh, as mattf suggested I think that we should have 0.7.0 release of client w/o backward compat aliaces 18:12:12 aliases* 18:12:35 I'm thinking about tomorrow if everything will work good 18:12:45 so, it's a good time to propose fixes to the client ;) 18:13:07 ok, let's move on 18:13:11 #topic RC1 status 18:13:33 so, as planned we postpone RC1 cut for one week and it'll be next week 18:14:00 we're now waiting for rest of Hadoop 2 changes to be merged for both vanilla and idh 18:14:22 + I hope to include doc updates to the RC1 too to avoid a big number of backports 18:14:41 #info RC1 next week, ~ April 3 18:14:46 any objections? 18:15:18 is there anyone here? 18:15:23 :) 18:15:26 here :) 18:15:28 no objection 18:15:30 no objection 18:15:38 good, cool 18:15:38 no objections :) 18:15:40 Ok w/ me 18:15:42 no objection 18:15:50 how much work do we have left for IDH 2.0? 18:16:00 I ask because of this 18:16:02 #info http://gigaom.com/2014/03/27/intel-ditches-its-hadoop-distro-and-puts-millions-behind-cloudera/ 18:16:03 * SergeyLukjanov fighting cold and just wake up :) 18:16:22 just change the I to C 18:16:22 * mattf is here now 18:16:31 jspeidel, it's ready and on review, just waiting for stable passing os savanna-ci 18:16:36 jspeidel, good point 18:16:39 ErikB1, :) 18:17:12 let's discuss idh integration later, I hope alazarev will share some thoughts / details on it 18:17:43 [22:12:08] oh, as mattf suggested I think that we should have 0.7.0 release of client w/o backward compat aliaces 18:17:43 [22:12:12] aliases* 18:17:43 [22:12:36] I'm thinking about tomorrow if everything will work good 18:17:47 matt ^^ 18:17:50 mattf* 18:18:17 ok, moving on 18:18:27 #topic Bug / doc days 18:18:29 SergeyLukjanov, ack, tomorrow, check 18:18:53 jspeidel: integration with IDH 2.0 is done, integration tests are on review 18:18:59 we should have a "testing" day after the RC1 to verify sahara 18:19:06 and re doc day - we need it 18:19:10 really need 18:19:16 alazarev, jspeidel is suggesting no one will care... intel has dropped idh 18:19:47 so, I'm proposing to choose a day early next week to write some docs 18:20:05 and we need volunteer for tracking docs, ruhe is unavailable 18:20:26 I'm proposing Monday for doc day 18:20:37 what does that exactly mean? 18:21:10 coordinate/collaborate to write docs, team will work on it the whole day 18:21:19 like we have bug triage day in the mid cycle 18:21:35 * ErikB1 thinks that sounds fine 18:21:49 it'll help to push missing docs faster 18:22:00 so, here is a blueprint https://blueprints.launchpad.net/sahara/+spec/update-docs-icehouse 18:22:04 +1 for Monday 18:22:13 any volunteers to track doc activity and status? 18:22:25 update blueprint, assign writers, erc. 18:22:44 :( 18:23:10 i will volunteer to help, but i don't have the most experience :/ 18:23:58 elmiko, you'll need to set find volunteers for doc updates and know their status + find missed items 18:24:21 that sounds... challenging 18:24:51 if other's are willing to help though, i don't mind trying to collate everything 18:24:52 elmiko, thanks for volunteering 18:25:07 jspeidel: :) 18:25:15 I'll certainly help update docs wherever I can 18:25:28 i say the next person to join does it 18:25:37 elmiko: I'll help you if i have time on Mon 18:25:39 I really hope that we will write most of the docs at Monday 18:25:48 lol 18:25:50 themistymay, winner! 18:26:06 yeah? 18:26:10 themistymay, congratulations! 18:26:15 just to be clear, based on the blueprint SergeyLukjanov posted i need to make sure we get all those items finished? 18:26:25 oh god, what did i get myself into 18:26:27 themistymay, by being the first person to join (last), you get the option to drive a docs day on monday 18:26:34 themistymay: you will coordinate doc updating on monday 18:26:36 themistymay, you're the chief doc writer of openstack! 18:26:40 * elmiko backs aways slowly 18:26:49 oh man 18:26:59 that sounds, like a blast 18:27:04 :) 18:27:24 sounds like no one is jumping at the chance 18:27:28 i get my opensouce approvals back and this is what i get 18:27:36 I'd be happy to help 18:27:39 elmiko, yup, and probably add some more items 18:27:43 but leading may be dangerous 18:28:00 so, I'm assigning bp to elmiko 18:28:11 i'll give it my best :) 18:28:11 and hope that we all help him on coordination 18:28:19 and hope all folks will write docs ;) 18:28:33 elmiko, I can help with HDP related docs 18:28:56 it doesn't mean that elmiko will write all docs :) 18:29:06 or even any 18:29:12 I hope some ;) 18:29:16 my only concern is my lack of familiarity with sahara 18:29:22 elmiko: probably I;m the right person for IDH stuff 18:29:24 same here 18:29:37 so, I'm expecting that component "owners" will cover their components like plugins, etc. 18:29:42 cool 18:29:47 actually, do we have any new text to be added to docs? 18:29:53 SergeyLukjanov, +1 18:29:58 i am logging this, so beware, i will find people on monday! 18:29:59 we should update all docs in fact 18:30:10 especcially dev / installation / user guides 18:31:04 docs are very important for us - it's our first real OpenStack aligned release 18:31:33 so, I'm asking all of you to write/check/update docs 18:31:40 can do! 18:31:56 any objects for doc day @ Monday? 18:32:18 no objections :) 18:32:22 none here 18:32:30 no objections 18:32:35 native writers are welcome to update mistakes in the docs :) 18:32:57 aignatov :P 18:33:02 yeah, that's a real issue for Russians ;) 18:33:06 late to the party, sure, I'll sign up 18:33:21 awesome 18:33:25 We missed the meeting last week and I forgot about it :) 18:33:32 tmckay, me too :) 18:33:33 Short memory 18:33:35 it was cancelled 18:33:52 ack, missed == cancelled 18:35:02 * SergeyLukjanov probably we'll unavailable on Tuesday - kitchen furniture and electronics "installation" :) 18:35:13 ok, let's move on 18:35:31 #topic Action items from the last meeting 18:35:41 #info nope 18:35:48 #topic News / updates 18:35:51 folks, please! 18:36:45 few of renaming, heat bug fixing were on me 18:36:49 last week 18:37:10 Mostly just small bugs for me on the dashboard recently. I'm also trying to come up to speed on disk image building. 18:37:47 I am back to available for now. My new company is reviewing the paperwork for intellectual property still, until they get back to me, I am free to commit to open source. If things change, I will update and drop off again (*sigh*). Docs seem like a good step. 18:37:50 #info http://gigaom.com/2014/03/27/intel-ditches-its-hadoop-distro-and-puts-millions-behind-cloudera/ <- news, raises question about future of idh plugin 18:38:09 mattf, yes 18:38:29 is this the correct time to discuss IDH? 18:38:33 Still working on Blueprints. Figuring out when we can integrate with HDP Plugin. 18:39:12 Also looking at logging and determining what the short comings are on Sahara when *certain* things fail and what is logged. Essentially a usability issue. 18:39:39 yup, we could try (re is this the correct time to discuss IDH?) 18:39:59 IDH integration is done, no sense in canceling it in Icehouse 18:40:09 alazarev, agreed 18:40:11 alazarev, agreed 18:40:28 I would vote to remove it after Icehouse 18:40:40 we should consider adding cloudera in J 18:41:11 ErikB1: did you think about demo examples? 18:41:31 and it would be great to have cloudera integration done by cloudera guy 18:41:31 so, re IDH - we should discuss removing of it with Intel folks first 18:42:01 and then make a decision on summit with the default answer to remove it in Juno release 18:42:07 alazarev, ++ 18:42:31 what's about integration tests for idh? 18:42:53 aignatov, I wouldn't spend any more time on them 18:43:25 aignatov, we should merge and run them for Icehouse due to the fact we're including plugin into the release 18:43:44 aignatov, yes keep whatever we have 18:43:53 ok 18:44:02 the funny thing is that we have a good enough coverage :) 18:44:17 s/funny/sad/ 18:44:30 I see, I meat that we can make idh job is non-voting 18:45:00 aignatov: I think we should keep it voting for Icehouse 18:45:05 aignatov: why? if we support plugin it should work 18:45:21 if we ship it in the release, then lets make sure it is working 18:45:26 yup, if we're including it in I then it should work 18:45:27 aignatov, +1 to keep it as voting 18:45:30 aignatov: unless we remove plugin completely 18:46:00 so, let's keep idh job voting till the Icehouse release and than will see 18:46:02 sounds business as usual w/ a slant to removing idh in J 18:46:13 lets just not spend any more time on it then we need to get existing tests stable 18:46:23 ok, ok, I'm surrendering, it was just an idea :) 18:46:26 but we should get feedback from idh plugin users -- how do we do that? 18:46:52 mattf, you need to find them first 18:47:20 #action alazarev contact intel folks to clarify idh drop off 18:47:31 probably there will be some migration plan from Intel 18:47:33 in a world where you don't know your users the best you can do is public execute a deprecation plan 18:47:57 I don't believe they will just stop IDH support 18:48:00 and doing that takes time, so something for J 18:48:27 mattf, yup, release in I and decide on summit for J sounds good for me 18:48:59 we should also discuss HDP1 and Vanilla1 18:49:10 mattf: agree 18:49:49 so, agreed on "IDH plugin release as part of Icehouse, decide the fate of IDH on summit and probably deprecate / remove in Juno" 18:49:51 as I see Hadoop 2.X is the trend now, so for me, we don't need to expand features on Hadoop 2.x 18:49:52 any objections? 18:50:08 SergeyLukjanov: +1 18:50:09 aignatov, you mean 1.x? 18:50:15 yes 18:50:28 SergeyLukjanov: +1 18:50:54 aignatov, that seems to be right 18:50:57 do we have Hadoop 2 images pre-generated anywhere? 18:51:12 I was playing with HDP plugin but I used centos with 1.3.2 18:51:24 #agreed IDH plugin release as part of Icehouse, decide the fate of IDH on summit and probably deprecate / remove in Juno 18:51:25 hadoop2 is a trend for customers who build new hadoop clouds 18:51:30 ErikB1, bob_nettleton ^^ 18:51:52 jspeidel, alazarev: well, it should be discussed on the summit 18:52:06 I'm afraid that there are still too many users of Hadoop 1, so, probably, it's a bit early to drop of 1.x 18:52:07 tmckay, we have HDP 2.0.6 images generated in sahara-image-elements 18:52:11 customers who have hadoop clouds already just don't look into sahara's side now 18:52:16 anyway, it's a good topic for summit 18:52:30 SergeyLukjanov, not drop just not spending time to enhance 18:52:41 bob_nettleton, okay, great 18:52:59 tkmckay, if you use "diskimage-create.sh -p hdp", the script will generate images for both HDP versions. 18:53:01 now that we have multiple plugins and the tech is moving, we should have a discussion about plugin lifecycle 18:53:33 jspeidel, sounds reasonable IMO 18:53:34 bob_nettleton, thank you 18:53:35 jspeidel: It is strange to add features to hadoop2 and don't support them in 1.x 18:53:36 something prematurely kicked off before hong kong 18:53:54 tmckay, you're welcome. 18:54:01 tmckay: IvanBerezovsky told me that he upload vanilla hadoop 2 images to the savanna-files 18:54:02 alazarev, yeah, feature parity / support from plugins... complex. 18:54:20 aignatov, that's great, too, thanks 18:54:41 mattf, yup, especially while we're growing our base features 18:55:10 5 min warning 18:55:13 aignatov, did we make any progress on picking a demo to do? I forgot to look in my training manuals ... 18:55:25 * mattf is very interested in ^^ 18:55:33 VERY 18:55:52 tmckay: no, i'm looking for ErikB1's ideas 18:55:54 mattf, missing the antecedent of ^^ 18:56:01 which ^^ is that? 18:56:03 we have recently seen issue with openstack rate limits when trying to provision cluster > 20 18:56:06 tmckay, demo 18:56:11 is this an issue with Heat integration? 18:56:12 thought so :) 18:56:42 jspeidel: did you use heat engine for your cluster? 18:56:46 mattf, I meant to ping jvyas and see if he has ideas 18:56:57 actually it's not turned on by default 18:57:02 #topic Open discussion 18:57:05 jspiedel: you can tune rate limits, is there any problem with that? 18:57:06 3 mins left 18:57:18 ^^ bnettleton 18:57:45 bob_nettleton: were you able to generate proper centos-based images? I've seen few issues ( elmiko too ), maybe selinux 18:57:50 one idea from the books it's a weather analyser job 18:58:02 jspeidel, anyway, it depends on OpenStack conf and yes, it could fail if rates are very low 18:58:20 tosky, I have been able to generate HDP images on Centos. I've been using Ubuntu to generate the images without any problems. 18:58:27 jspeidel, but in the heat provisioning engine, it should b handled by heat 18:58:57 SergeyLukjanov, is heat turned on by default in Sahara now? 18:59:03 aignatov, an accurate weather predictor would be awesome 18:59:06 Sergeyukjanov, appears that the limits are very low by default 18:59:08 1 min left 18:59:11 bob_nettleton: so, there is an issue when generating them from Fedora or CentOS (see https://bugs.launchpad.net/sahara/+bug/1292614 ) 18:59:14 bob_nettleton, nope 18:59:26 oh, good catch 18:59:34 bob_nettleton: and also, the CentOS image that I generated from Ubuntu needed a selinux relabeling 18:59:40 tmckay: one concern is that canonical example from the book 18:59:52 looks like we should postpone enabling heat by default to the time when heat will gate on sahara to avoid random issues 18:59:54 it's kinda of boring for hadoopers :) 18:59:58 aignatov, ack 19:00:01 like we see last month 19:00:04 SergeyLukjanov, +1 19:00:08 #endmeeting