Tuesday, 2013-10-01

« Monday, 2013-09-30 Index Wednesday, 2013-10-02 »
*** kbroughton has joined #savanna00:20
*** nosnos has joined #savanna00:50
*** IlyaE has joined #savanna02:40
*** SergeyLukjanov has joined #savanna05:08
*** SergeyLukjanov has quit IRC05:14
*** SergeyLukjanov has joined #savanna05:15
*** SergeyLukjanov has quit IRC06:15
*** SergeyLukjanov has joined #savanna07:18
*** DinaBelova has joined #savanna07:19
*** alazarev has joined #savanna07:28
*** alazarev has quit IRC07:35
*** kbroughton has quit IRC07:44
*** alazarev has joined #savanna07:52
*** DinaBelova has quit IRC08:00
*** DinaBelova has joined #savanna08:08
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Fix python-savannaclient version in reqs  https://review.openstack.org/4896109:20
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna-dashboard: Migrate to using python-savannaclient 0.3a2  https://review.openstack.org/4860709:21
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna-dashboard: Remove usage of pyvirtualdisplay  https://review.openstack.org/4911809:22
openstackgerritA change was merged to stackforge/python-savannaclient: Bump oslo.config to >=1.2.0  https://review.openstack.org/4906109:40
openstackgerritA change was merged to stackforge/savanna: Removing line breaks from default configs  https://review.openstack.org/4906509:42
*** DinaBelova has quit IRC09:59
*** SergeyLukjanov has quit IRC10:25
*** SergeyLukjanov has joined #savanna10:27
*** DinaBelova has joined #savanna10:30
*** alazarev has quit IRC10:55
*** kbroughton has joined #savanna11:00
*** alazarev has joined #savanna11:09
*** alazarev has quit IRC11:11
*** nosnos has quit IRC11:33
openstackgerritYaroslav Lobankov proposed a change to stackforge/savanna: Migration to new integration tests  https://review.openstack.org/4865211:36
*** matsuhashi has quit IRC11:37
*** alazarev has joined #savanna11:58
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Migration to new integration tests  https://review.openstack.org/4865212:04
openstackgerritNikita Konovalov proposed a change to stackforge/python-savannaclient: Fix config handling for Node Group Templates  https://review.openstack.org/4914512:17
*** alazarev has quit IRC12:28
*** _crobertsrh is now known as crobertsrh12:29
openstackgerritA change was merged to stackforge/python-savannaclient: Fix config handling for Node Group Templates  https://review.openstack.org/4914512:30
*** alazarev has joined #savanna12:31
openstackgerritNikita Konovalov proposed a change to stackforge/savanna-dashboard: Migrate to using python-savannaclient 0.3a3  https://review.openstack.org/4860712:39
*** NikitaKonovalov has joined #savanna12:40
openstackgerritNikita Konovalov proposed a change to stackforge/savanna: Migration to new integration tests  https://review.openstack.org/4865212:46
*** _mattf is now known as mattf12:57
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Add admin context for non-request ops  https://review.openstack.org/4914913:03
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Impl multitenancy support  https://review.openstack.org/4915013:03
*** kbroughton has quit IRC13:18
*** kbroughton has joined #savanna13:34
openstackgerritA change was merged to stackforge/savanna-dashboard: Migrate to using python-savannaclient 0.3a3  https://review.openstack.org/4860713:38
openstackgerritA change was merged to stackforge/savanna: Added missed default configs for Oozie-4.0.0  https://review.openstack.org/4908413:48
*** tmckay has joined #savanna13:58
tmckayfreenode webchat client, cool13:58
tmckayI'm behind a web service at a car service place and pidgin can't seem to make it through (I think it's the "Accept" screen)13:59
tmckaygit review doesn't seem to work either :)14:00
openstackgerritDmitry Mescheryakov proposed a change to stackforge/savanna: Implement threaded SSH for provisioning and Vanilla plugin  https://review.openstack.org/4864614:03
*** tmckay has quit IRC14:09
*** IlyaE has quit IRC14:21
*** tmckay has joined #savanna14:21
*** IlyaE has joined #savanna14:22
*** tmckay has quit IRC14:34
*** tmckay has joined #savanna14:34
*** tmckay has quit IRC14:58
*** DinaBelova has quit IRC15:04
*** tmckay has joined #savanna15:04
openstackgerritNikita Konovalov proposed a change to stackforge/savanna-dashboard: Fix EDP launch Job Execution  https://review.openstack.org/4917015:30
openstackgerritA change was merged to stackforge/savanna-dashboard: Fix EDP launch Job Execution  https://review.openstack.org/4917015:39
*** NikitaKonovalov has left #savanna15:40
*** SergeyLukjanov has quit IRC15:54
*** mattf is now known as _mattf16:11
*** SergeyLukjanov has joined #savanna16:20
*** DinaBelova has joined #savanna16:20
*** DinaBelova has quit IRC16:20
*** DinaBelova has joined #savanna16:38
openstackgerritA change was merged to stackforge/savanna: Migration to new integration tests  https://review.openstack.org/4865216:59
tmckaySergeyLukjanov, ping re 49150 and tenant_id17:07
*** sacharya has joined #savanna17:27
tmckayaignatov, ping17:30
openstackgerritA change was merged to stackforge/savanna: Add admin context for non-request ops  https://review.openstack.org/4914917:37
SergeyLukjanovtmckay, hi17:37
SergeyLukjanovtmckay, do you have questions about 49150?17:38
tmckaySergeyLukjanov, hi, you already answered them in comments :)  Thanks17:39
SergeyLukjanovtmckay, ok :)17:39
tmckayI pinged you here before I wrote comments17:39
SergeyLukjanovoh, sorry for delay17:39
tmckayoh, no problem, I assumed you were otherwise occupied so I used gerrit instead17:39
tmckaySergeyLukjanvo, since you are here...  on swift data retrieval, there is a facility that we can use (chunk_size) so that the object returned internal_swift.py are generators, not entire files as strings17:41
tmckaySergeyLukjanov, I mean ^^17:41
*** _mattf is now known as mattf17:42
tmckayI am considering whether or not to put in a "throttle" value, so that if the data returned for an entire file would be > some maximum it returns the generator from swiftclient instead.17:42
tmckayThis would protect against someone uploading a 5GB file to swift and then referencing it from a JobBinary and retrieving it (either through the API or a job run).  That might cause memory issues on the host.17:43
tmckaySeems safer.  A caller would have to test whether the result was a generator or a string.  This above is my question for aignatov, too -- do you think it is worth the effort to code this?17:45
tmckayThe JobManager would be an example of a consumer that would loop through the generate piece by piece instead of loading the whole thing in memory at once.17:46
* tmckay knows QE guys who try to break things like this all the time17:47
aignatovtmckay, actually I didn't understand actual your question on the review. Yes, defenitely you should expose some limit to protect uploading binaries to swift or internal db17:53
aignatovsorry for delay17:54
tmckayaignatov, well, we can't limit swift uploads from savanna.  We can limit what we read...17:54
tmckayaignatov, my thought was this.  Instead of saying "that file is too big, you cannot retrieve it from swift...."17:54
SergeyLukjanovtmckay, mm, I think that it's not very important atm17:55
tmckayif we use the generator facility in the swiftclient we can handle a large file in pieces and avoid saying "no" while limiting memory usage17:55
SergeyLukjanovtmckay, I hope that we'll switch from our current rest api solution to OpenStack-approved solution like Pecan and WSME17:55
*** DinaBelova has quit IRC17:55
SergeyLukjanovand then we can impl theottling17:55
tmckaySergeyLukjanov, aignatov, I am thinking of the JobManager, too, in addition to REST.  The JobManager retrieves binaries as strings and then transfers them to hdfs.17:56
aignatovso, Sergey you mean that we can add big TODO to the code: "Here we should think and Implement throttling"?17:56
tmckaySo, that string exists in memory in Python17:57
SergeyLukjanovyep, I think that we should add TODO17:57
tmckayMaybe for now we should just have a configurable limit.  No retrieval of binary bigger than ??MB17:58
SergeyLukjanovsuch throttling is needed not for the all providers17:58
SergeyLukjanovas tmckay said it could be handled ok with swiftclient17:58
tmckayI'll add a size check with a configurable value to the draft CR, how about that?  And the community can comment :)18:00
aignatovI'm ok with that.18:00
tmckayokay, thanks.  I had bad dreams of the JobManager crashing ;-)18:01
*** DinaBelova has joined #savanna18:04
openstackgerritSergey Lukjanov proposed a change to stackforge/savanna: Impl multitenancy support  https://review.openstack.org/4915018:06
SergeyLukjanovtmckay, it's ok for me too, just thinking about code/feature freeze18:10
*** alazarev has quit IRC18:13
tmckaySergeyLukjanov, ah, yes.  When is that?18:14
SergeyLukjanovhttp://lists.openstack.org/pipermail/openstack-dev/2013-September/015800.html18:15
tmckaythanks :)  I should read openstack-dev more18:15
*** rnirmal has joined #savanna18:31
SergeyLukjanovtmckay, btw you can subscribe only to [savanna] marked emails18:32
tmckaySergeyLukjanov, that is a great idea18:33
SergeyLukjanovthe rest emails could be important too, so, it's hard to choose...18:33
SergeyLukjanovpersonally, I'm reading the hole ML18:33
tmckayack, I would like to pay attention to all of it.  Maybe 2 folders...  I should be able to implement the size check today, I see how to do it in the swiftclient, and publish the CR this evening.  Plenty of time for the freez.18:34
*** DinaBelova has quit IRC18:44
*** DinaBelova has joined #savanna19:11
*** DinaBelova has quit IRC19:24
*** aignatov has quit IRC19:27
*** DinaBelova has joined #savanna19:28
openstackgerritChad Roberts proposed a change to stackforge/savanna-dashboard: Adding help text to EDP forms  https://review.openstack.org/4924319:35
*** alazarev has joined #savanna19:37
openstackgerritChad Roberts proposed a change to stackforge/savanna-dashboard: Adding help text to EDP forms  https://review.openstack.org/4924319:54
*** DinaBelova has quit IRC20:01
*** alazarev has quit IRC20:04
*** rnirmal has quit IRC20:08
*** aignatov has joined #savanna20:08
openstackgerritTrevor McKay proposed a change to stackforge/savanna: Allow job binaries to be retrieved from internal swift  https://review.openstack.org/4875120:54
*** IlyaE has quit IRC21:03
SergeyLukjanovfor your info folks, we'll move savanna to openstack org this weekend21:06
SergeyLukjanovoct 5 1600 utc21:07
crobertsrh:)21:08
*** crobertsrh is now known as _crobertsrh21:08
tmckayyay21:08
*** IlyaE has joined #savanna21:18
*** IlyaE has quit IRC21:25
*** mattf is now known as _mattf21:29
*** sacharya has quit IRC21:49
*** IlyaE has joined #savanna21:51
*** SergeyLukjanov has quit IRC21:54
*** SergeyLukjanov has joined #savanna21:58
*** SergeyLukjanov has quit IRC22:01
*** tmckay has left #savanna22:35
*** tmckay has joined #savanna22:36
*** kbroughton has quit IRC22:50
*** sacharya has joined #savanna23:07
*** IlyaE has quit IRC23:29
*** IlyaE has joined #savanna23:40
« Monday, 2013-09-30 Index Wednesday, 2013-10-02 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!