Thursday, 2013-08-01

« Wednesday, 2013-07-31 Index Friday, 2013-08-02 »
*** IlyaE has joined #savanna00:11
*** IlyaE has quit IRC00:21
*** IlyaE has joined #savanna00:39
*** IlyaE has quit IRC00:50
*** dina_belova has joined #savanna00:50
*** NikitaKonovalov has joined #savanna00:54
*** dina_belova has quit IRC00:55
*** IlyaE has joined #savanna00:57
*** NikitaKonovalov has quit IRC00:58
*** IlyaE has quit IRC01:05
*** dina_belova has joined #savanna01:52
*** NikitaKonovalov has joined #savanna01:55
*** dina_belova has quit IRC01:56
*** NikitaKonovalov has quit IRC01:59
*** SergeyLukjanov has joined #savanna02:50
*** dina_belova has joined #savanna02:51
*** NikitaKonovalov has joined #savanna02:56
*** NikitaKonovalov has quit IRC03:00
*** SergeyLukjanov has quit IRC03:05
*** dina_belova has quit IRC03:15
*** IlyaE has joined #savanna03:54
*** NikitaKonovalov has joined #savanna03:56
*** NikitaKonovalov has quit IRC04:01
*** IlyaE has quit IRC04:15
*** IlyaE has joined #savanna04:19
*** NikitaKonovalov has joined #savanna04:57
*** _mattf is now known as mattf05:00
*** NikitaKonovalov has quit IRC05:01
*** dina_belova has joined #savanna05:15
*** IlyaE has quit IRC05:15
*** dina_belova has quit IRC05:16
*** lastidiot has quit IRC05:20
*** dina_belova has joined #savanna05:25
*** tmckayrh has quit IRC05:26
*** IlyaE has joined #savanna05:37
*** dina_belova has quit IRC05:52
*** aignatov has quit IRC06:04
*** aignatov has joined #savanna06:04
*** NikitaKonovalov has joined #savanna06:11
*** dina_belova has joined #savanna06:20
*** dina_belova has quit IRC06:25
*** mattf has quit IRC06:44
*** dina_belova has joined #savanna06:44
*** dina_belova has quit IRC06:50
*** NikitaKonovalov has quit IRC06:53
*** dina_belova has joined #savanna06:55
*** IlyaE has quit IRC06:58
*** NikitaKonovalov has joined #savanna07:04
*** dina_belova has quit IRC07:08
*** aignatov has quit IRC07:09
*** IlyaE has joined #savanna07:09
*** aignatov has joined #savanna07:10
*** IlyaE has quit IRC07:14
*** NikitaKonovalov has quit IRC07:28
*** NikitaKonovalov has joined #savanna07:47
*** nprivalova has joined #savanna07:56
*** dina_belova has joined #savanna07:59
*** SergeyLukjanov has joined #savanna07:59
*** NikitaKo_ has joined #savanna08:02
*** NikitaKonovalov has quit IRC08:04
*** NikitaKo_ has quit IRC08:04
*** NikitaKonovalov has joined #savanna08:04
*** dina_belova has quit IRC08:10
openstackgerritNikita Konovalov proposed a change to stackforge/python-savannaclient: Python Savanna Client initial implementation  https://review.openstack.org/3826108:14
*** dina_belova has joined #savanna08:28
*** ruhe has joined #savanna08:44
*** dmitryme has joined #savanna08:56
openstackgerritNikita Konovalov proposed a change to stackforge/savanna-dashboard: Fix skipping node_group_id if not specified  https://review.openstack.org/3965208:56
openstackgerritNikita Konovalov proposed a change to stackforge/savanna-dashboard: Fix skipping node_group_id if not specified  https://review.openstack.org/3965208:57
openstackgerritA change was merged to stackforge/savanna: Ambari install screen after install fix  https://review.openstack.org/3937509:12
openstackgerritA change was merged to stackforge/savanna-dashboard: Fix skipping node_group_id if not specified  https://review.openstack.org/3965209:23
*** NikitaKonovalov has quit IRC09:33
*** NikitaKonovalov has joined #savanna09:36
*** NikitaKonovalov has quit IRC09:42
*** dina_belova has quit IRC09:57
*** ruhe has quit IRC10:04
*** nprivalova has quit IRC10:22
*** SergeyLukjanov has quit IRC10:41
*** NikitaKonovalov has joined #savanna10:42
*** ruhe has joined #savanna10:45
*** NikitaKonovalov has quit IRC10:46
*** nprivalova has joined #savanna10:52
openstackgerritAlexander Ignatov proposed a change to stackforge/savanna: Resolves critical issue with oozie service  https://review.openstack.org/3966610:57
*** dina_belova has joined #savanna10:58
*** dina_belova has quit IRC11:02
*** NikitaKonovalov has joined #savanna11:10
*** dina_belova has joined #savanna11:11
*** SergeyLukjanov has joined #savanna11:11
openstackgerritAlexander Ignatov proposed a change to stackforge/savanna-extra: Added Oozie DIB element  https://review.openstack.org/3967111:26
openstackgerritAlexander Ignatov proposed a change to stackforge/savanna-extra: Added Oozie DIB element  https://review.openstack.org/3967111:28
openstackgerritAlexander Ignatov proposed a change to stackforge/savanna-extra: Added Oozie DIB element  https://review.openstack.org/3967111:36
*** dmitryme has quit IRC12:06
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Initial part of conductor implementation  https://review.openstack.org/3927012:22
*** ruhe has quit IRC12:32
*** NikitaKonovalov has quit IRC12:43
*** NikitaKonovalov has joined #savanna12:47
*** dmitryme has joined #savanna12:50
*** ruhe has joined #savanna12:54
*** _tstclair is now known as tstclair12:58
*** ruhe has quit IRC12:59
*** _mattf has joined #savanna13:00
*** _mattf is now known as mattf13:01
*** mattf has joined #savanna13:01
*** dmitryme has quit IRC13:32
*** tmckayrh has joined #savanna13:33
tmckayrhSergeyLukjanov, fyi, just because I mentioned it yesterday -- the http connection to the Swift REST api did not work because apparently whatever httpie is doing to resolve host names does not work with "localhost".  I've had similar issues in Python before, I believe (something to do with httplib or urllib).13:37
openstackgerritJonathan Maron proposed a change to stackforge/savanna: Fix to convert parsing failure  https://review.openstack.org/3955313:48
*** NikitaKonovalov has quit IRC13:53
*** dmitryme has joined #savanna13:58
*** SergeyLukjanov has quit IRC14:01
*** NikitaKonovalov has joined #savanna14:02
*** NikitaKonovalov has quit IRC14:15
*** lastidiot has joined #savanna14:22
openstackgerritSergey Reshetnyak proposed a change to stackforge/savanna: Refactoring remote utils  https://review.openstack.org/3904314:23
*** dmitryme_ has joined #savanna14:29
*** dmitryme has quit IRC14:31
*** dmitryme_ is now known as dmitryme14:31
*** lastidiot has quit IRC14:34
*** dina_belova has quit IRC14:39
*** NikitaKonovalov has joined #savanna14:46
*** NikitaKonovalov has quit IRC14:55
*** ruhe has joined #savanna14:57
*** SergeyLukjanov has joined #savanna14:59
*** lastidiot has joined #savanna15:09
*** NikitaKonovalov has joined #savanna15:16
openstackgerritSergey Reshetnyak proposed a change to stackforge/savanna: Refactoring hdp plugin  https://review.openstack.org/3973315:18
*** dina_belova has joined #savanna15:19
openstackgerritA change was merged to stackforge/savanna: Initial part of conductor implementation  https://review.openstack.org/3927015:22
*** rnirmal has joined #savanna15:37
*** dina_belova has quit IRC15:47
*** SergeyLukjanov has quit IRC15:48
*** NikitaKonovalov has quit IRC15:50
*** dmitryme has quit IRC15:55
*** dmitryme has joined #savanna15:57
openstackgerritSergey Reshetnyak proposed a change to stackforge/savanna: Refactoring remote utils  https://review.openstack.org/3904315:59
*** IlyaE has joined #savanna16:01
*** ruhe has quit IRC16:01
*** dmitryme has quit IRC16:19
*** dina_belova has joined #savanna16:20
*** SergeyLukjanov has joined #savanna16:25
*** dmitryme has joined #savanna16:26
*** dmitryme has quit IRC16:38
*** ruhe has joined #savanna16:39
*** NikitaKonovalov has joined #savanna16:46
*** NikitaKonovalov has quit IRC16:51
*** mattf is now known as _mattf16:52
*** _mattf is now known as mattf16:54
*** ruhe has quit IRC16:56
*** ben_duyujie has joined #savanna17:03
*** NikitaKonovalov has joined #savanna17:13
*** nprivalova has quit IRC17:27
*** ben_duyujie has quit IRC17:31
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Port sqlalchemy db models to conductor  https://review.openstack.org/3975617:31
*** ben_duyujie has joined #savanna17:33
*** dina_belova has quit IRC17:43
*** IlyaE has quit IRC17:45
*** Nadya has joined #savanna17:52
SergeyLukjanovsavanna meeting will be in #openstack-meeting-alt channel in 7 minutes17:58
*** dmitryme has joined #savanna17:58
*** aignatov_ has joined #savanna18:02
*** NikitaKonovalov has quit IRC18:02
*** NikitaKonovalov has joined #savanna18:02
*** benl_ has joined #savanna18:05
*** Nadya_ has joined #savanna18:05
*** akuznetsov has joined #savanna18:05
*** Nadya has quit IRC18:06
*** IlyaE has joined #savanna18:07
*** ben_duyujie has quit IRC18:08
*** benl__ has joined #savanna18:11
*** benl_ has quit IRC18:12
*** ruhe has joined #savanna18:19
*** ruhe_ has joined #savanna18:36
*** ruhe has quit IRC18:37
*** dina_belova has joined #savanna18:43
*** _crobertsrh has quit IRC18:45
*** dina_belova has quit IRC18:49
*** dina_belova has joined #savanna18:54
SergeyLukjanovbenl__, what's your lp id?19:03
SergeyLukjanov(to assign PKI bp to you)19:03
benl__Hmm ben-lapid I think19:04
*** tmckayrh is now known as tmckay_latelunch19:04
SergeyLukjanovdone - https://blueprints.launchpad.net/savanna/+spec/pki-support19:05
benl__Cool, so whats the current situation? Savanna works with the old Tokens and not PKI?19:05
SergeyLukjanovbenl__, yep19:06
SergeyLukjanovbenl__, we are using keystone client's middleware for auth19:06
benl__Okay. I'll start digging into it19:07
SergeyLukjanovand afaik it should provide auth for both uuid and poi tokens...19:07
benl__Hmm, Shouldn't that support PKI out of the box?19:07
SergeyLukjanovbenl__, great, thank you!19:07
SergeyLukjanovbenl__, I think it should:)19:07
benl__I see, I'll investigate :)19:08
*** aignatov_ has quit IRC19:11
tmckay_latelunchNadya_, re your comment "<Nadya_> tmckayrh, if you have questions about swift integration please post your question in irc. I dealt with swift in savanna"19:17
*** tmckay_latelunch is now known as tmckay19:17
*** tmckay is now known as tmckayrh19:17
*** NikitaKonovalov has quit IRC19:18
Nadya_yes, I'm hrer19:19
Nadya_*here19:19
tmckayrhNadya_, I am wondering what swift integration there is in savanna already.  I installed swift as part of an openstack instance on a development machine19:20
*** NikitaKonovalov has joined #savanna19:21
tmckayrhah, I found the swift subdir...19:21
Nadya_oh, It's a long story :) The thing is swift is not integrated with hadoop. To make them work together it is needed to put jar file with swift FileSystem impl  to hadoop libs19:22
tmckayrhI think I saw some comments about that...19:22
*** akuznetsov has quit IRC19:22
Nadya_did you find docs about it?19:22
tmckayrhI forget where.... something mentioned it, with links to issues19:23
tmckayrhswift+hadoop19:23
Nadya_give me a second I will find19:24
tmckayrhBut, whatever packstack did on my box, swift is running at least well enough to interact with using the python client or http so I thought that would be adequate for API development19:24
*** benl__ has quit IRC19:24
Nadya_http://savanna.readthedocs.org/en/latest/userdoc/hadoop-swift.html19:27
tmckayrhah yes, that's it19:27
*** dina_belova has quit IRC19:29
Nadya_could you please explain what kind of integration are you implementing? what part? pushing jobs "binaris" to and from swift?19:30
*** benl_ has joined #savanna19:31
dmitrymetmckayrh19:32
Nadya_patching is needed only for interacting with Hadoop. e.g. if you want to run job and get data from swift19:32
dmitrymetmckayrh: here are some more words about integration with Swift19:32
dmitrymehttps://savanna.readthedocs.org/en/latest/userdoc/features.html19:32
dmitrymeIn particular, here is described how one should compose Swift URL19:33
tmckayrhNadya_, yes, I am working on implementing the Job Origin (formerly Job Source) api in savanna.  So, the first part will be registering sources.  There is supposed to be a call to retrieve a job, though -- that will ultimately be a download from swift19:34
tmckayrh"registering origins" excuse me19:34
tmckayrhNadya_, as of yet, the blueprint doesn't say anything about pushing jobs, only retrieving them19:34
benl_Nadya_, is there a particular reason to patch hadoop instead of using a Swift filesystem over FUSE?19:34
tmckayrhSo I'm assuming that someone would push jobs to swift externally with another client19:35
*** NikitaKonovalov has quit IRC19:35
*** NikitaKonovalov has joined #savanna19:36
tmckayrhSwift is the first job origin type to support.  Then of course there are the scm variants, and internal savanna storage, etc.  I started with swift on Sergey's suggestion (I think it was him :)  Maybe ruhe)19:36
Nadya_tmckayrh, are you going to copy jobs from swift to hdfs?19:36
*** rnirmal has quit IRC19:36
Nadya_I mean jobs' "binaries"19:36
tmckayrhNadya_, not sure.  I haven't dived into the details of "get a binary" too much yet.  Still learning savanna internals.19:37
*** benl__ has joined #savanna19:38
tmckayrhcould be, I suppose.  The sequence diagram is very high level -- what does "Job Manager retrieves binary" really mean?19:38
tmckayrhNadya_, maybe we should chat about that.  How will your Oozie flows want to see binaries?  Where will it look?19:39
Nadya_tmckayrh, let's assume we have pig script. May it be on swift? And what will we do with it? I thought that on this itegation we are not working with this scenario19:40
dmitrymebenl_: for Vanilla plugin we are preparing an image with Hadoop anyway. So it is easier to put patch here at build time19:40
Nadya_tmckayrh, the working scenario is only "data to be processes is on swift"19:40
benl__I see. cheers19:40
tmckayrhNadya_, so in the case of an Ooozie workflow.xml, where would you want it to be?19:41
tmckayrhhdfs?19:42
*** benl_ has quit IRC19:42
Nadya_tmckayrh, it should be in HDFS I believe. We thought about variant of storing it into swift but not sure if we have time to implement it19:42
*** dina_belova has joined #savanna19:44
tmckayrhNadya_, okay, I can do that.  Is the difficulty with using Swift the integration that you pointed out earlier?19:44
*** ruhe_ has quit IRC19:45
Nadya_tmckayrh, it is absolutly  possible to store scripts and workflow xml in swift. The problem is only in resources to implement it. Yes, problem is in integration. Anyway, we will think about it19:47
tmckayrhokay19:48
Nadya_ruhe managed to run swift-located scripts19:48
Nadya_but he didn't tell me how :)19:48
tmckayrh:)  I just played with the swift cli for upload/download, I thought "Great!"19:49
tmckayrhI can store it, and I can read it back.19:49
Nadya_yes, it is. but it is not so simple for hadoop scenario :) e.g. try to copy from swift to hdfs19:50
mattfSergeyLukjanov, IlyaE, fyi, i added some placeholders to the roadmap page19:50
tmckayrhNadya_, thank you for pointing this out.19:50
SergeyLukjanovmattf, yep, see it, thank you19:51
*** benl__ has quit IRC19:52
SergeyLukjanovmattf, I think that incubation request should removed from the list of tech improvements and ubuntu should be added to the integration target19:52
Nadya_tmckayrh, you are welcome! anyway, I will evaluate the complexity of running scripts and jars directly from swift. Maybe it is scaring only at first glance19:54
tmckayrhNadya_, haha, that is my trouble -- I don't know enough yet to be scared :)  Still on the "peaches and rose petals" part of the learning curve19:55
Nadya_tmckayrh, :)19:57
mattfSergeyLukjanov, where would you put the statement about when we'd like to submit an incubation request?20:02
mattfSergeyLukjanov, good call wrt ubuntu, slipped my mind20:02
*** dina_belova has quit IRC20:06
*** benl_ has joined #savanna20:07
*** dina_belova has joined #savanna20:08
*** Nadya_ has quit IRC20:13
*** crobertsrh has joined #savanna20:15
*** crobertsrh has left #savanna20:17
*** crobertsrh has joined #savanna20:17
*** dina_belova has quit IRC20:18
*** rnirmal has joined #savanna20:19
*** IlyaE has quit IRC20:33
*** IlyaE has joined #savanna20:50
*** mattf is now known as _mattf20:59
*** SergeyLukjanov has quit IRC21:00
*** benl_ has quit IRC21:01
*** IlyaE has quit IRC21:06
*** benl_ has joined #savanna21:06
*** NikitaKonovalov has quit IRC21:12
crobertsrhSorry for bailing on the meeting earlier.  My internet connection dropped just after the "Job Origin" vote.  I went ahead and created the blueprint for edp-savanna-dashboard-ui.  I can add more detail if desired https://blueprints.launchpad.net/savanna/+spec/edp-savanna-dashboard-ui21:18
*** tstclair is now known as _tstclair21:20
*** dmitryme has quit IRC21:21
*** openstackgerrit has quit IRC21:25
*** openstackgerrit has joined #savanna21:25
benl_Hey, anyone around? I could use some help in setting up savanna21:28
*** dina_belova has joined #savanna21:29
crobertsrhbenl_:  Have you checked out https://savanna.readthedocs.org/en/latest/21:29
benl_Yup, followed everything21:30
crobertsrhThen you're all set, right? :)21:30
benl_I'm having the same issue as this guy: https://lists.launchpad.net/savanna-all/msg00028.html21:30
benl_Yet when I try to do the 'service-create' thing, I get  "Unable to authorize user"21:30
benl_(I exported OS_USERNAME and OS_PASSWORD to the admin user, and OS_AUTH_URL to port 35357)21:31
crobertsrhah, that was my next question21:31
benl_:)21:31
benl_That should be a part of https://savanna.readthedocs.org/en/latest/horizon/installation.guide.html right?21:32
benl_or is it only in specific setups?21:32
benl_(Im using devstack, with PKI instead of UUID and some neutron, ceilometer and heat stuff enabled)21:33
crobertsrhI'm afraid I might not be of much help.  I just started working with savanna recently.  I haven't had a lot of exposure to every part of the system yet.21:33
*** dina_belova has quit IRC21:33
benl_Thats alright, thanks anyway21:34
crobertsrhHopefully, someone with more expertise can chime in21:34
crobertsrhGood luck21:36
*** crobertsrh is now known as _crobertsrh21:36
*** ben_duyujie has joined #savanna21:47
*** NikitaKonovalov has joined #savanna22:13
*** rnirmal has quit IRC22:15
*** NikitaKonovalov has quit IRC22:17
*** lastidiot has quit IRC22:21
*** aignatov has quit IRC22:25
*** aignatov has joined #savanna22:26
*** nmakhotkin has quit IRC22:28
*** benl_ has quit IRC22:32
*** benl_ has joined #savanna22:32
*** nmakhotkin has joined #savanna22:42
*** IlyaE has joined #savanna22:45
*** IlyaE has quit IRC22:52
*** IlyaE has joined #savanna22:53
*** IlyaE has quit IRC22:55
*** benl_ has quit IRC22:55
*** NikitaKonovalov has joined #savanna23:13
*** NikitaKonovalov has quit IRC23:18
*** benl_ has joined #savanna23:26
*** benl_ has quit IRC23:33
*** _mattf is now known as mattf23:51
*** lastidiot has joined #savanna23:52
« Wednesday, 2013-07-31 Index Friday, 2013-08-02 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!