Monday, 2013-08-12

*** dina_belova has joined #savanna00:04
*** dina_belova has quit IRC00:09
*** NikitaKonovalov has joined #savanna00:41
*** NikitaKonovalov has quit IRC00:45
*** dina_belova has joined #savanna01:05
*** dina_belova has quit IRC01:10
*** IlyaE has joined #savanna01:37
*** NikitaKonovalov has joined #savanna01:41
*** NikitaKonovalov has quit IRC01:46
*** dina_belova has joined #savanna02:05
*** dina_belova has quit IRC02:10
*** IlyaE has quit IRC02:24
*** SergeyLukjanov has joined #savanna02:34
*** NikitaKonovalov has joined #savanna02:42
*** NikitaKonovalov has quit IRC02:47
*** dina_belova has joined #savanna03:06
*** dina_belova has quit IRC03:10
*** SergeyLukjanov has quit IRC03:21
*** NikitaKonovalov has joined #savanna03:43
*** NikitaKonovalov has quit IRC03:47
*** akuznetsov has quit IRC04:37
*** akuznetsov has joined #savanna04:38
*** NikitaKonovalov has joined #savanna04:43
*** NikitaKonovalov has quit IRC04:47
*** bmahe has quit IRC04:49
*** dina_belova has joined #savanna05:09
*** dina_bel_ has joined #savanna05:14
*** dina_belova has quit IRC05:14
*** dina_belova has joined #savanna05:21
*** dina_be__ has joined #savanna05:23
*** dina_bel_ has quit IRC05:23
*** dina_belova has quit IRC05:27
*** SergeyLukjanov has joined #savanna05:28
*** SergeyLukjanov has quit IRC05:37
*** SergeyLukjanov has joined #savanna05:42
*** NikitaKonovalov has joined #savanna05:44
*** NikitaKonovalov has quit IRC05:48
*** ben_duyujie has joined #savanna05:51
*** SergeyLukjanov has quit IRC06:05
*** dina_be__ has quit IRC06:23
*** SergeyLukjanov has joined #savanna06:27
*** dina_belova has joined #savanna06:28
*** NikitaKonovalov has joined #savanna06:44
*** SergeyLukjanov has quit IRC06:48
*** NikitaKonovalov has quit IRC06:49
*** SergeyLukjanov has joined #savanna06:56
*** SergeyLukjanov has quit IRC07:01
*** dina_belova has quit IRC07:05
*** IlyaE has joined #savanna07:09
*** dina_belova has joined #savanna07:13
*** akuznetsov has quit IRC07:19
*** SergeyLukjanov has joined #savanna07:20
*** NikitaKonovalov has joined #savanna07:25
*** NikitaKonovalov has quit IRC07:29
*** SergeyLukjanov has quit IRC07:35
*** bogdando has joined #savanna07:41
*** IlyaE has quit IRC07:41
*** dina_belova has quit IRC07:49
*** NikitaKonovalov has joined #savanna07:49
*** dina_belova has joined #savanna07:49
*** ben_duyujie has quit IRC07:55
*** ben_duyujie has joined #savanna07:56
*** NikitaKonovalov has quit IRC08:20
*** dina_belova has quit IRC08:32
*** nprivalova has joined #savanna08:33
*** dina_belova has joined #savanna08:33
openstackgerritNadya Privalova proposed a change to stackforge/savanna: Added conductor API for Job Object  https://review.openstack.org/4111908:43
*** SergeyLukjanov has joined #savanna08:44
*** NikitaKonovalov has joined #savanna08:50
*** SergeyLukjanov has quit IRC08:55
*** SergeyLukjanov has joined #savanna08:55
*** NikitaKonovalov has quit IRC08:59
openstackgerritNadya Privalova proposed a change to stackforge/savanna: Added conductor API for Job Object  https://review.openstack.org/4111908:59
*** akuznetsov has joined #savanna09:01
*** ruhe has joined #savanna09:08
*** akuznetsov has quit IRC09:14
*** ruhe has quit IRC09:24
openstackgerritA change was merged to stackforge/savanna: Added conductor API for DataSource object  https://review.openstack.org/4059109:29
*** dina_belova has quit IRC09:37
*** ruhe has joined #savanna09:49
openstackgerritA change was merged to stackforge/savanna: Conductor objects are re-populated on update  https://review.openstack.org/4112209:50
*** NikitaKonovalov has joined #savanna09:50
*** akuznetsov has joined #savanna09:51
*** NikitaKonovalov has quit IRC09:55
*** dina_belova has joined #savanna09:56
*** akuznetsov has quit IRC10:06
*** ruhe has quit IRC10:09
*** SergeyLukjanov has quit IRC10:16
*** akuznetsov has joined #savanna10:17
*** ruhe has joined #savanna10:18
*** SergeyLukjanov has joined #savanna10:18
*** NikitaKonovalov has joined #savanna10:21
*** NikitaKonovalov has quit IRC10:26
*** qwerty_nor has joined #savanna10:31
*** nprivalova has quit IRC10:38
*** NikitaKonovalov has joined #savanna10:53
*** akuznetsov has quit IRC10:54
*** NikitaKonovalov has quit IRC10:57
*** akuznetsov has joined #savanna10:58
*** nprivalova has joined #savanna11:02
openstackgerritNikolay Mahotkin proposed a change to stackforge/savanna: Added Hive configuration, new nodeprocess - hiveserver  https://review.openstack.org/4084711:10
*** NikitaKonovalov has joined #savanna11:24
*** NikitaKonovalov has quit IRC11:29
openstackgerritA change was merged to stackforge/savanna: Added conductor API for Job Object  https://review.openstack.org/4111911:31
*** NikitaKonovalov has joined #savanna11:55
*** NikitaKonovalov has quit IRC11:59
*** akuznetsov has quit IRC12:05
openstackgerritDmitry Mescheryakov proposed a change to stackforge/savanna: Conductor API re-init only objects, not IDs  https://review.openstack.org/4139312:07
*** akuznetsov has joined #savanna12:20
openstackgerritDmitry Mescheryakov proposed a change to stackforge/savanna: Conductor API re-init only objects, not IDs  https://review.openstack.org/4139312:24
*** NikitaKonovalov has joined #savanna12:26
*** NikitaKonovalov has quit IRC12:30
*** dina_belova has quit IRC12:42
*** dina_belova has joined #savanna12:44
*** dina_belova has quit IRC12:45
*** ogelbukh has joined #savanna12:47
_crobertsrhQuestion all:  I've got some UI stuff done for EDP.  Even though it isn't fully wired-in yet, I'd like to get it out there.  Should I just prepare a draft review for it, or should I just go ahead an submit it as a "real" review as a work in progress?12:51
mattfif it is self-contained and doesn't break master (configured off by default?), you should just file a review -- with the idea that more, smaller reviews are better than fewer, larger reviews12:54
SergeyLukjanovhi12:56
ruhei'd also suggest to add akuznetsov and dmitrymex to reviewers list12:56
SergeyLukjanovit'll be better to publish change request12:56
*** NikitaKonovalov has joined #savanna12:56
_crobertsrhNo, it adds things to the savanna panel that aren't fully working yet.  Nothing breaks, but often nothing is really created yet.12:57
SergeyLukjanovit could be in WIP status to make reviewers able to place some early comments12:58
_crobertsrhOk.  I will fire up a WIP change request just so its out there.  Probably sometime later this morning.  Thanks.12:59
*** NikitaKonovalov has quit IRC13:01
akuznetsov_crobertsrh now we have a savanna-pythonclient, possibly you should have look to it. SergeyLukjanov, when we intend to use it in horizon dashboard?13:02
_crobertsrhYeah, good point.  I saw that it was created.  I will take a look at it.13:02
SergeyLukjanovakuznetsov, we have no plans for upgrading savanna dashboard to use savanna client for the next several weeks13:05
ruheseems like a good place for _crobertsrh to contribute :)13:06
_crobertsrhFor sure.13:06
*** _crobertsrh is now known as crobertsrh13:06
*** akuznetsov has quit IRC13:09
*** tmckayrh has joined #savanna13:11
*** akuznetsov has joined #savanna13:23
*** ben_duyujie has quit IRC13:24
*** NikitaKonovalov has joined #savanna13:27
*** tstclair has joined #savanna13:30
*** NikitaKonovalov has quit IRC13:31
*** ruhe has quit IRC13:45
openstackgerritDmitry Mescheryakov proposed a change to stackforge/savanna: Conductor API re-init only objects, not IDs  https://review.openstack.org/4139313:48
openstackgerritJohn Speidel proposed a change to stackforge/savanna: Allow Ambari users to be specified in configuration  https://review.openstack.org/4114613:54
*** NikitaKonovalov has joined #savanna13:58
*** ruhe has joined #savanna13:58
crobertsrhakuznetsov:  In the schema for job (edp/job.py), it looks like URL is being required, which I don't think is correct.  Would you agree?  If I try to submit something with a URL, I get the "extra properties are not permitted" error.14:00
*** dina_belova has joined #savanna14:01
*** dmitryme has joined #savanna14:02
*** NikitaKonovalov has quit IRC14:03
akuznetsovcrobertsrh this is mistake, I am fixed it when will work on integration REST API and DB layer14:04
crobertsrhOk.  Just making sure I wasn't misunderstanding something.  Thanks.14:05
openstackgerritIvan Berezovskiy proposed a change to stackforge/savanna-extra: Add hive element  https://review.openstack.org/4142614:05
*** dina_belova has quit IRC14:06
openstackgerritIvan Berezovskiy proposed a change to stackforge/savanna-extra: Add hive element  https://review.openstack.org/4142614:07
*** dina_belova has joined #savanna14:11
*** dina_belova has quit IRC14:16
*** ruhe has quit IRC14:18
openstackgerritAlexander Ignatov proposed a change to stackforge/savanna: Oozie bug-fixing  https://review.openstack.org/4143214:20
openstackgerritDenis Egorenko proposed a change to stackforge/savanna-extra: Add element savanna-version that write commit id into the image  https://review.openstack.org/4143314:21
*** openstackgerrit has quit IRC14:24
*** openstackgerrit has joined #savanna14:24
*** NikitaKonovalov has joined #savanna14:30
*** NikitaKonovalov has quit IRC14:34
*** dina_belova has joined #savanna14:34
*** ruhe has joined #savanna14:36
*** openstackgerrit has quit IRC14:40
*** openstackgerrit has joined #savanna14:40
*** dina_belova has quit IRC14:47
*** dina_belova has joined #savanna14:48
*** rnirmal has joined #savanna15:00
*** rnirmal has quit IRC15:00
*** rnirmal has joined #savanna15:00
*** NikitaKonovalov has joined #savanna15:00
*** NikitaKonovalov has quit IRC15:05
*** akuznetsov has quit IRC15:07
*** akuznetsov has joined #savanna15:10
*** ruhe has quit IRC15:21
openstackgerritNadya Privalova proposed a change to stackforge/savanna: Added conductor API for JobExecutor Object  https://review.openstack.org/4144815:22
*** IlyaE has joined #savanna15:25
*** IlyaE has quit IRC15:26
*** IlyaE has joined #savanna15:26
*** ruhe has joined #savanna15:29
*** dina_belova has quit IRC15:29
*** NikitaKonovalov has joined #savanna15:32
*** dina_belova has joined #savanna15:36
*** NikitaKonovalov has quit IRC15:36
*** dina_bel_ has joined #savanna15:37
*** dina_belova has quit IRC15:37
openstackgerritNadya Privalova proposed a change to stackforge/savanna: Added conductor API for JobExecution Object  https://review.openstack.org/4144815:39
tmckayrhupdate... I've got basic JobOrigin data objects working.... I just have to make the responses look like the "clusters" stuff, and I should probably add unit tests.15:44
tmckayrh"copy" does nothing yet -- that will be the next step, to move binaries from one source to another.  Actually, maybe an api to store binaries in the internal savanna db first...15:44
*** sacharya has joined #savanna15:48
*** ruhe has quit IRC15:55
*** akuznetsov has quit IRC15:55
*** akuznetsov has joined #savanna16:01
*** NikitaKonovalov has joined #savanna16:02
*** dmitryme_ has joined #savanna16:03
*** dmitryme has quit IRC16:05
*** dmitryme_ is now known as dmitryme16:05
*** IlyaE has quit IRC16:07
*** NikitaKonovalov has quit IRC16:07
*** NikitaKonovalov has joined #savanna16:09
*** dmitryme has quit IRC16:12
*** NikitaKonovalov has quit IRC16:18
*** NikitaKonovalov has joined #savanna16:19
*** NikitaKonovalov has quit IRC16:26
*** dmitryme has joined #savanna16:27
*** NikitaKonovalov has joined #savanna16:27
*** ruhe has joined #savanna16:28
nprivalovatmckayrh, "api to store binaries in the internal savanna db" - I strongly agree with it16:29
tmckayrhnprivalova, ok16:32
*** akuznetsov has quit IRC16:34
aignatovtmckayrh, hi, regarding "copy" method... i'm think about getting rid of it :)16:36
*** NikitaKonovalov has quit IRC16:37
openstackgerritYaroslav Lobankov proposed a change to stackforge/savanna: Integration test updating for "HDP" plugin  https://review.openstack.org/4036916:38
tmckayrhaignatov, okay.  If we have binaries in swift, do we run from swift?16:38
tmckayrhAnd what about binaries in the savanna db?16:38
aignatovif it doesn't contain any functional and there are no plans to implement in hte closest future may be we should leave JobOrigin without it16:38
tmckayrhif we don't have copy, does that mean that in the first pass we can store binaries in hdfs only, and then run directly from that hdfs?16:39
tmckayrhbrb16:39
*** akuznetsov has joined #savanna16:39
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Add check for deprecated method assertEquals  https://review.openstack.org/4146616:39
* tmckayrh tmckayrh_errand16:40
*** NikitaKonovalov has joined #savanna16:41
aignatovah, your copy metthod is to copy original binaries for example from internal savanna-db to provisioned hdfs?16:41
aignatovright?16:41
*** nprivalova has quit IRC16:41
*** NikitaKonovalov has quit IRC16:42
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Fix test files names  https://review.openstack.org/4146716:42
*** dina_bel_ has quit IRC16:42
aignatovi'm just not sure if this method should have separate REST call16:42
*** NikitaKonovalov has joined #savanna16:43
openstackgerritA change was merged to stackforge/savanna: Conductor API re-init only objects, not IDs  https://review.openstack.org/4139316:43
aignatovmaybe JobManager will copy binaries and put it into HDFS intrnally, so JobManager should know how to copy binaries16:44
*** ruhe has quit IRC16:44
aignatovakuznetsov, what do you think anout above our discussions?16:44
aignatov*about16:44
akuznetsovJobManager know where to copy, but functionality for copy can be in JobOrigin component16:46
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Fix retrieval of updated id  https://review.openstack.org/4146816:46
aignatovand should it be called via savanna REST api?16:47
akuznetsovno16:48
aignatovso we don't need it in savanna/api/v11.py16:49
aignatovright?16:49
SergeyLukjanovI16:51
SergeyLukjanovI agree that we don't need copy in api now16:51
*** NikitaKonovalov has quit IRC16:51
*** NikitaKonovalov has joined #savanna16:52
*** dina_belova has joined #savanna16:53
*** NikitaKonovalov has quit IRC16:53
*** IlyaE has joined #savanna16:55
*** NikitaKonovalov has joined #savanna17:04
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Add check for deprecated method assertEquals  https://review.openstack.org/4146617:06
openstackgerritA change was merged to stackforge/savanna: Allow Ambari users to be specified in configuration  https://review.openstack.org/4114617:06
*** dina_belova has quit IRC17:08
*** SergeyLukjanov has quit IRC17:09
*** dmitryme has quit IRC17:13
*** dmitryme has joined #savanna17:16
*** NikitaKonovalov has quit IRC17:21
*** NikitaKonovalov has joined #savanna17:22
*** akuznetsov has quit IRC17:24
openstackgerritJonathan Maron proposed a change to stackforge/savanna: Made Ambari RPM location configurable  https://review.openstack.org/4047917:29
*** NikitaKonovalov has quit IRC17:31
*** shredder12 has joined #savanna17:32
*** shredder12 has joined #savanna17:32
*** NikitaKonovalov has joined #savanna17:32
shredder12hi mattf17:35
shredder12i've gone through the documentation on readthedocs. I've installed and setup the environment, but I am yet to integrate savanna with openstack. I'll start working on hadoop v2.0 soon. My understanding is that you are looking for a separate plugin to handle v2.0.17:38
dmitrymeshredder12: yep, we think that a separate plugin should be developed for v2.0, but we also want v1 and v2 plugins to share as much code as possible17:44
*** NikitaKonovalov has quit IRC17:44
*** NikitaKonovalov has joined #savanna17:45
shredder12hi dmitryme  I've gone through vanilla plugin's code.  I think I'll start with integrating savanna with the my local openstack(folsom). Swift is updated to grizzly so new features critical to Hadoop should work.17:49
tmckayrhaignatov, okay, I see the earlier discussion.  I'll remove copy17:49
shredder12dmitryme, I'm yet to try v2. Once the setup is up and running, I'll get started with it.17:50
dmitrymeshredder12: great to hear that, thanks for the update!17:51
*** ruhe has joined #savanna17:52
*** IlyaE has quit IRC17:57
*** IlyaE has joined #savanna17:58
openstackgerritChad Roberts proposed a change to stackforge/savanna-dashboard: Adds EDP support in the UI  https://review.openstack.org/4148218:01
crobertsrhmattf:  Initial set of UI changes for EDP ^^^18:01
*** SergeyLukjanov has joined #savanna18:03
ruhecrobertsrh awesome18:03
crobertsrhStill plenty to do, but it's a start18:03
*** ruhe has quit IRC18:04
*** dmitryme has quit IRC18:10
*** tmckayrh has quit IRC18:10
*** tmckayrh has joined #savanna18:12
*** rodarvus has left #savanna18:15
tmckayrhcrobertsrh, fyi -- the copy op will be removed from the JobOrigin rest api,18:16
tmckayrhnot sure if you hooked up to it18:16
crobertsrhyeah, saw that.  no prob.  never hooked up18:16
*** dina_belova has joined #savanna18:28
*** IlyaE has quit IRC18:28
tmckayrhSergeyLukjanov, aignatov, I have database objects for JobOrigin.  Should I extend the api review https://review.openstack.org/#/c/40936/ to include them, or do it as a separate change request?18:33
SergeyLukjanovtmckayrh, I think it'll be easier to handle several CRs instead of big one18:34
tmckayrhokay, thanks.  I was wary of expanding the scope of an existing CR :)18:34
*** NikitaKonovalov has quit IRC18:45
*** NikitaKonovalov has joined #savanna18:46
*** openstackgerrit has quit IRC18:50
*** openstackgerrit has joined #savanna18:50
openstackgerritChad Roberts proposed a change to stackforge/savanna-dashboard: Adds EDP support in the UI  https://review.openstack.org/4148218:52
*** IlyaE has joined #savanna18:53
*** NikitaKonovalov has quit IRC18:53
*** NikitaKonovalov has joined #savanna18:54
openstackgerritTrevor McKay proposed a change to stackforge/savanna:     Add initial version of the REST api for the job origin component  https://review.openstack.org/4093619:00
*** ruhe has joined #savanna19:08
*** ruhe has quit IRC19:32
mattfhttp://koji.fedoraproject.org/koji/taskinfo?taskID=5809350 <- wooo19:34
openstackgerritA change was merged to stackforge/savanna: Refactoring hdp plugin  https://review.openstack.org/3973319:37
*** dmitryme has joined #savanna19:37
crobertsrhnice mattf!19:41
*** akuznetsov has joined #savanna20:00
*** akuznetsov has quit IRC20:07
*** dina_belova has quit IRC20:16
tmckayrhSergeyLukjanov, or others, I have this from git because I am trying to add the sql object for joborigin as a separate change, from the same topic branch.  Is it okay to say "yes"?20:18
tmckayrh$ git review20:18
tmckayrhYou are about to submit more than one commit.20:18
tmckayrhThe outstanding commits are:20:18
tmckayrhf0dac88 (HEAD, bp/edp-job-origin-component) Add JobOrigin sqlite object20:18
tmckayrh3ec6881     Add initial version of the REST api for the job origin component20:18
tmckayrhhow do we make successive changes on the same topic branch, with multiple changes in review at the same time?  Or does gerrit not support that20:19
SergeyLukjanovif you doesn't update the old commit than nothing happens with it20:21
SergeyLukjanovonly new commit will be uploaded20:21
tmckayrhokay, that sounds good :)20:22
tmckayrhThe "Are you sure??" gave me pause.20:22
openstackgerritTrevor McKay proposed a change to stackforge/savanna: Add JobOrigin sqlite object  https://review.openstack.org/4151420:22
SergeyLukjanovlooks like it works ok for you :)20:23
tmckayrh yes, and it's smart enough to get the dependency :)  Yay.20:24
tmckayrhthat was my next step, to figure out how to add the dep20:24
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Migrate to pbr  https://review.openstack.org/3748020:27
tmckayrhSergeyLukjanov, so far I created that JobOrigin object in the old db model -- I should implement in db_new as well, I suppose.20:28
*** NikitaKonovalov has quit IRC20:28
tmckayrhIs there a convenient way to test db_new implementations?  (should I just change the api code to call it instead?)20:29
SergeyLukjanovwe're planning to replace old db with new one in a few days20:30
SergeyLukjanovI hope tomorrow :)20:30
tmckayrhso I should create the classes?20:30
SergeyLukjanovthere a lot of unit tests for the new db in savanna/tests/unit/conductor20:30
tmckayrhokay, I'll see if I can hijack some of those20:30
SergeyLukjanovI think that you can move your db code to the db_new20:31
SergeyLukjanovyou can take a look at akuznetsov patch for example20:31
* tmckayrh thinks maybe he shouldn't use that word in modern times ^^20:31
openstackgerritChad Roberts proposed a change to stackforge/savanna-dashboard: Adds EDP support in the UI  https://review.openstack.org/4148220:34
*** crobertsrh is now known as _crobertsrh20:38
*** SergeyLukjanov has quit IRC20:51
*** dmitryme has quit IRC21:10
*** dina_belova has joined #savanna21:17
*** dina_belova has quit IRC21:22
*** dina_belova has joined #savanna21:27
*** NikitaKonovalov has joined #savanna21:29
*** tstclair is now known as _tstclair21:31
*** dina_belova has quit IRC21:31
*** NikitaKonovalov has quit IRC21:33
*** _tstclair has quit IRC21:37
*** openstackgerrit has quit IRC22:24
*** openstackgerrit has joined #savanna22:25
*** dina_belova has joined #savanna22:28
*** NikitaKonovalov has joined #savanna22:29
*** dina_belova has quit IRC22:34
*** NikitaKonovalov has quit IRC22:34
*** sacharya has quit IRC22:47
*** rnirmal has quit IRC23:06
*** dina_belova has joined #savanna23:28
*** NikitaKonovalov has joined #savanna23:29
*** dina_belova has quit IRC23:33
*** NikitaKonovalov has quit IRC23:33

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!