Thursday, 2013-05-16

*** yidclare has quit IRC00:22
*** yamahata__ has quit IRC00:26
*** cp16net is now known as cp16net|away00:28
*** cp16net|away is now known as cp16net00:28
*** IlyaE has joined #openstack-meeting-alt00:31
*** sacharya1 has joined #openstack-meeting-alt00:38
*** sacharya has quit IRC00:40
*** bdpayne has quit IRC00:42
*** yamahata__ has joined #openstack-meeting-alt00:44
*** IlyaE has quit IRC01:26
*** markwash has joined #openstack-meeting-alt01:39
*** markwash has quit IRC01:51
*** yamahata__ has quit IRC01:53
*** yamahata__ has joined #openstack-meeting-alt02:05
*** IlyaE has joined #openstack-meeting-alt02:17
*** amyt has joined #openstack-meeting-alt02:44
*** amyt has quit IRC02:45
*** amyt has joined #openstack-meeting-alt02:45
*** clarkb has quit IRC03:13
*** chmouel has quit IRC03:13
*** HenryG_ has joined #openstack-meeting-alt03:15
*** chmouel has joined #openstack-meeting-alt03:15
*** HenryG has quit IRC03:19
*** SergeyLukjanov has joined #openstack-meeting-alt04:09
*** bdpayne has joined #openstack-meeting-alt04:30
*** asalkeld has joined #openstack-meeting-alt04:49
*** asalkeld has left #openstack-meeting-alt04:57
*** cp16net is now known as cp16net|away04:57
*** SergeyLukjanov has quit IRC05:06
*** sacharya1 has quit IRC05:07
*** clarkb has joined #openstack-meeting-alt05:11
*** bdpayne has quit IRC05:26
*** SergeyLukjanov has joined #openstack-meeting-alt05:55
*** aababilov has joined #openstack-meeting-alt06:00
*** amyt has quit IRC06:07
*** cp16net|away is now known as cp16net06:07
*** aababilov has left #openstack-meeting-alt06:25
*** SergeyLukjanov has quit IRC06:59
*** IlyaE has quit IRC07:00
*** dmitryme has joined #openstack-meeting-alt08:31
*** dmitryme has quit IRC08:58
*** SergeyLukjanov has joined #openstack-meeting-alt09:11
*** thatsdone has joined #openstack-meeting-alt10:43
*** qwerty_nor has joined #openstack-meeting-alt10:59
*** dmitryme has joined #openstack-meeting-alt11:21
*** thatsdone has quit IRC11:25
*** SergeyLu_ has joined #openstack-meeting-alt11:38
*** SergeyLukjanov has quit IRC11:41
*** dmitryme has quit IRC11:47
*** cloudchimp has joined #openstack-meeting-alt12:01
*** sergmelikyan has joined #openstack-meeting-alt12:03
*** dhellmann-away is now known as dhellmann12:06
*** rnirmal has joined #openstack-meeting-alt12:13
*** dmitryme has joined #openstack-meeting-alt12:29
*** dhellmann has quit IRC12:31
*** hartsocks has joined #openstack-meeting-alt12:38
*** yidclare has joined #openstack-meeting-alt12:40
*** dmitryme has quit IRC12:42
*** dosaboy_ has joined #openstack-meeting-alt12:45
*** dmitryme has joined #openstack-meeting-alt12:52
*** dosaboy_ has quit IRC12:53
*** amyt has joined #openstack-meeting-alt13:05
*** SergeyLu_ has quit IRC13:08
*** jcru has joined #openstack-meeting-alt13:37
*** amyt has quit IRC13:46
*** amyt has joined #openstack-meeting-alt13:47
*** SergeyLukjanov has joined #openstack-meeting-alt13:47
*** qwerty_nor has quit IRC13:56
*** qwerty_nor has joined #openstack-meeting-alt13:59
*** sacharya has joined #openstack-meeting-alt14:02
*** dhellmann has joined #openstack-meeting-alt14:10
*** djohnstone has joined #openstack-meeting-alt14:11
*** djohnstone has quit IRC14:11
*** djohnstone has joined #openstack-meeting-alt14:11
*** sacharya has quit IRC14:16
*** IlyaE has joined #openstack-meeting-alt14:33
*** grapex has joined #openstack-meeting-alt14:37
*** cp16net is now known as cp16net|away14:50
*** amyt has quit IRC14:53
*** sacharya has joined #openstack-meeting-alt14:54
*** amyt has joined #openstack-meeting-alt14:54
*** jeblair has quit IRC15:00
*** amyt has quit IRC15:01
*** jeblair has joined #openstack-meeting-alt15:01
*** amyt has joined #openstack-meeting-alt15:02
*** amyt has quit IRC15:05
*** amyt has joined #openstack-meeting-alt15:06
*** cp16net|away is now known as cp16net15:15
*** HenryG_ has quit IRC15:18
*** bdpayne has joined #openstack-meeting-alt15:23
*** cp16net is now known as cp16net|away15:30
*** cp16net|away is now known as cp16net15:30
*** markwash has joined #openstack-meeting-alt15:49
*** amyt has quit IRC15:50
*** amyt has joined #openstack-meeting-alt15:50
*** yidclare has quit IRC15:51
*** cloudchimp has quit IRC15:53
*** markwash has quit IRC15:57
*** markwash has joined #openstack-meeting-alt15:59
*** sarob has joined #openstack-meeting-alt16:01
*** amyt has quit IRC16:02
*** amyt has joined #openstack-meeting-alt16:02
*** grapex has quit IRC16:03
*** HenryG has joined #openstack-meeting-alt16:15
*** sarob_ has joined #openstack-meeting-alt16:21
*** sarob has quit IRC16:24
*** cp16net is now known as cp16net|away16:25
*** cp16net|away is now known as cp16net16:25
*** sarob_ has quit IRC16:29
*** sarob has joined #openstack-meeting-alt16:30
*** sarob has quit IRC16:36
*** sarob has joined #openstack-meeting-alt16:37
*** yidclare has joined #openstack-meeting-alt16:38
*** sarob has quit IRC16:42
*** dmitryme has quit IRC16:44
*** grapex has joined #openstack-meeting-alt16:45
*** SergeyLukjanov has quit IRC16:48
*** cp16net is now known as cp16net|away16:51
*** jrodom has joined #openstack-meeting-alt16:53
*** cp16net|away is now known as cp16net16:53
*** esp has joined #openstack-meeting-alt17:01
*** esp has left #openstack-meeting-alt17:15
*** SergeyLukjanov has joined #openstack-meeting-alt17:28
*** notmyname has quit IRC17:35
*** notmyname has joined #openstack-meeting-alt17:36
*** cp16net is now known as cp16net|away17:41
*** dmitryme has joined #openstack-meeting-alt17:47
*** IlyaE has quit IRC17:51
SergeyLukjanovHi everybody, Savanna meeting will start in a few minutes18:00
*** akuznetsov has joined #openstack-meeting-alt18:00
*** jspeidel has joined #openstack-meeting-alt18:01
*** jmaron has joined #openstack-meeting-alt18:01
*** amyt has quit IRC18:01
*** amyt has joined #openstack-meeting-alt18:02
*** ErikB has joined #openstack-meeting-alt18:02
*** aignatov2 has joined #openstack-meeting-alt18:02
ErikBHi Everyone18:03
SergeyLukjanovo/18:03
ErikBJust fired off a mail with a doc and a UI mock we can have a look at during the meeting.18:03
SergeyLukjanovare there everyone for savanna meeting?18:04
SergeyLukjanovok, lets start the meeting18:05
SergeyLukjanov#startmeeting savanna18:05
openstackMeeting started Thu May 16 18:05:16 2013 UTC.  The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot.18:05
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.18:05
*** openstack changes topic to " (Meeting topic: savanna)"18:05
openstackThe meeting name has been set to 'savanna'18:05
ErikBseems mail is taking a while. Here are the 2 docs just posed to the wiki: https://wiki.openstack.org/w/images/0/00/Hadoop_Configuration_Flow_For_Mockup.pdf and https://wiki.openstack.org/w/images/9/99/Hadoop_cluster_configuration_mockup.pdf18:06
*** cp16net|away is now known as cp16net18:06
SergeyLukjanovI think we have no agenda for the todays meeting18:07
ErikBCan we discuss the two docs just posted - concerning advanced confif?18:07
ErikBs/fif/fig18:07
*** Nadya has joined #openstack-meeting-alt18:08
SergeyLukjanovwe have a lot of discussions of plugin mechanism and Savanna's overall arch last week18:08
SergeyLukjanovErikB, yes, sure18:08
SergeyLukjanovgive us a moment to take a look on them18:08
*** ruhe has joined #openstack-meeting-alt18:09
*** mattf has joined #openstack-meeting-alt18:11
*** IlyaE has joined #openstack-meeting-alt18:11
SergeyLukjanovfor newcomers - here are the channel logs - http://eavesdrop.openstack.org/irclogs/%23openstack-meeting-alt/%23openstack-meeting-alt.2013-05-16.log18:12
*** highlycaffeinate has joined #openstack-meeting-alt18:12
dmitrymeErik, in your UI mocks. Does the Node Group section appears after user selects the file?18:12
jspeidelthe advanced file is optional18:12
jspeideleither way it is populated from result of plugin.populate_node_groups18:13
jspeidelcould be using default configuration18:13
ruheit looks the same as what Dmitry sent about three hours ago? just with different layout :)18:14
jmaronkind of..yes ;)18:14
ruhecool18:14
ErikBit was put together independently of what Dmitry did - so that is good that we are thinking the same18:15
ruheright18:15
rnirmalsorry guys got in a little late and may have to leave early as well18:16
jspeidelkeep in mind that we were not trying to design the UI, just provided the mockup for reasons of discussion18:17
jmaronin both cases, the selection of the template (standard or advanced) yields a view of the node groups18:17
jspeidelto better explain the flow18:17
ruhein the Object model changes. there is new field "vm_requirements". is it intended to help user to select appropriate flavor?18:17
jspeidelyes18:17
SergeyLukjanovam I right that "vm_requirements" is the minimal requirements for vms in node group?18:17
jspeidelyes18:17
rnirmalwe may be better of not looking at the UI first  ;)18:17
jspeidelthe associated flow doc describes how the ui uses vm_requirements18:18
jmaronnode count, cardinality, memory etc18:18
jspeidelit will allow the UI to do some validation18:18
rnirmal+1 for the vm requirements part... makes for good validation18:19
jspeideland only populate legal value18:19
jspeidelvalus18:19
*** dhellmann has quit IRC18:19
SergeyLukjanovjspiedel, yep, I see it18:19
rnirmalso let me bring up my concern from earlier about the plugin choosing a vm to use for a role18:19
*** dhellmann has joined #openstack-meeting-alt18:20
SergeyLukjanovour main point is to use one cluster object in all plugin api calls18:20
jmaronin the current proposed scheme, the plugin simply provides guidance via the requirements.  it's not choosing18:20
jspeidelSergeyLukjanov, agreed18:21
jmaronyes. we are following the same approach for advanced config18:21
ErikBThe controller chooses placement of VM18:21
rnirmalok that's good18:21
rnirmalalso in the future it would be good to have discussions around how the interactions happen via the api rather than the UI18:22
jspeidelrnirmal, you main the REST API?18:22
rnirmalmake the API simple enough and the UI will naturally flow in18:22
SergeyLukjanov"vm_requirements" looks good, I think that we should include such ability to specify minimal requirements for node groups18:22
rnirmaljspeidel: yes18:22
jspeidelyes, will need to discuss ramifications to REST API18:23
jspeidelbut lets settle on SPI first18:23
rnirmalsure18:23
jspeidelbetween controller and plugin18:23
jmaron+118:23
SergeyLukjanovyes, we should discuss the SPI first18:24
jspeidelfor the vm_requirements, we would need to have a defined set of properties that could be included18:24
jspeidelthat both the plugin and controller would understand18:24
jspeidelfor the UI to actually do any validation18:24
jmarona minimal set of expected values.  there could always be more for specific functions18:24
jspeidelotherwise it is still useful, but more of guidance to the user18:25
SergeyLukjanovjspeidel, all supported properties of flavor object plus cardinality18:25
jspeidel+ default_count18:25
rnirmalwhat you'll have there looks good, just addition of vcpus would finish it up18:25
jspeidelgood18:26
rnirmalnot sure about the default_count18:26
jspeidelit would allow users to "clone" an existing cluster18:26
rnirmalso if the user doesn't specify a count for that node-group is the default_count used18:26
jspeidelis just used to pre-populate vm count18:27
jmaronexactly18:27
jspeidelcan still be changed by user18:27
ErikBdefault_count is just what is recommended and can be changed by the user in the UI (it would be the default rendered in the UI)18:27
jspeidelit is just the initial value18:27
dmitrymemmm, why can't we just use existing 'count' field for that?18:27
rnirmalbut what about from the API18:27
jspeidellet us look ...18:27
rnirmalyou don't get that sort of an interaction from the rest api18:27
rnirmalI'd suggest not have the default_count for now18:27
SergeyLukjanovI think "default_count" is the very simple thing, lets discuss more important things for now18:28
jspeidelSergeyLukjanov, agreed18:28
jmaronand yes "count" may server the purpose18:28
jmaronserve18:28
jspeidelwe can discuss later after spi is nailed down18:28
SergeyLukjanovare there any concerns in SPI?18:29
SergeyLukjanovany other*18:29
*** dhellmann is now known as dhellmann-away18:29
jspeidelare you ok with the flow that we just posted with the mockup?18:30
rnirmalis anyone else working on a plugin other than ambari18:30
jmaronand the modified APIs?18:30
jmaronI had a thread on the list about the user_input object, but it a given input can reference it's source config object that should be more than sufficient18:30
jmaronits18:31
SergeyLukjanovjspeidel we published our new vision several ours ago and I think that we have the very similar vision18:31
ErikBrnirmal, not as far I know18:31
*** openstack has joined #openstack-meeting-alt18:33
*** ChanServ sets mode: +o openstack18:33
SergeyLukjanovadditionally, we have several concerns of some names of functions/approaches and etc.18:34
jspeideljmaron, wouldn't populate_node_groups still drive this?18:34
jspeidelthe plugin could use default values?18:34
SergeyLukjanov"convert" in our vision is very similar to yours "populate_node_groups"18:35
SergeyLukjanovbut we want to rename it18:35
jspeidelok, not too worried about naming18:35
jspeidelas long as we agree on functionality18:35
SergeyLukjanovyep ;)18:35
jspeideland that we don't need both18:35
jmaronI'm not sure.  populate_node_groups gets a cluster object that either references a provider template or a standard template.  In the provider case, obviously it's the plugin's responsibility.  But a standard cluster template could potentially understand the node groups defined in the standard template and pre-populate the node goups18:35
jmarongroups18:36
jspeideland that it works for both standard/advanced configs18:36
SergeyLukjanovbtw we want to not name file-based configuration as "advanced", because in our talks we name vms/hdfs placement as advanced cluster configuration18:36
ruhestandard flow doesn't need populate_node_groups. node groups are populated from the cluster template18:36
jspeidelplugin should always define node groups18:37
jspeidelwill be very different between vendors18:37
ruhetemplates will be plugin specific18:37
*** hub_cap has left #openstack-meeting-alt18:38
jspeidelhmmm.  ok, I guess that I am confused about templates18:38
ruhe*standard templates18:38
jspeidelI thought that they were just configuration not topology18:38
SergeyLukjanovand node template, for example, will define configs for the node group and plugin could override anything in "get_infra" function of the standard flow18:38
jspeidelwhy have two ways of exposing node groups18:38
jspeidel?18:38
ruheyou mean get_infra and and populate_node_groups are the same?18:39
jmaronlet's talk about a standard flow18:40
ruheok18:40
jmaronwhen a user selects the standard template in your mockup on the first screen18:40
jspeidelI am really confused about templates and how they relate to node groups18:40
jspeidelI thought that templates were just config n/v pairs18:41
jmaronis that template a template the plugin provided with a standardized set of node groups?18:41
dmitrymewe are speaking about Savanna native templates, they will look similar for all the providers18:43
dmitrymeand yes, previously we told that they will be key-value pairs18:43
jmaronand they will be pre-defined?  created by users?18:43
dmitrymethey will be created by users18:44
jmaronso they are a mechanism for defining node groups, correct?18:44
SergeyLukjanovbut I think that we can implement an ability for plugin to providing default templates18:45
dmitrymeright now we are thinking that cluster template should include several node templates, which will serve as templates for node groups18:45
dmitrymei.e. cluster template will include topology18:45
SergeyLukjanovone node template will describe configs and hardware for only one node group18:46
SergeyLukjanovand node group is the list of processes that will work at this node18:46
jspeidelI think that jmaron just explained how cluster/node templates to me.  Is there a doc that explains how users create templates?18:48
jspeidelIt is hard for me to get a good understanding over chat18:49
dmitrymeJohn, no, we don't have the doc with all that18:49
jmaronit would also be good if we started seeing some sample templates to get a better sense of how they work18:49
SergeyLukjanovit's outdated :( we planing to update it asap18:49
jspeidelI think that is critical18:49
*** flaper87 has joined #openstack-meeting-alt18:49
jspeidelat least for my understanding18:49
jspeidelcan Mirantis provide this soon?18:50
SergeyLukjanovin our current vision plugin should not know anything about templates18:50
dmitrymeyes, it is pretty important and we're going to work on it next18:50
SergeyLukjanovplugin will receive aggregated info (Cluster object)18:50
dmitrymeat the same time, plugin will not work directly with templates18:50
dmitrymeplugin will receive just list of configs18:50
jspeidelok, to be clear.  I am still a bit confused about how cluster/node templates relate to node groups.  So it is important that it is explained in the doc18:51
jspeideland how the plugin is involved in the creation of templates18:51
jmarontrue.  the doc you generate should list the plugin API calls (get_supported_node_processes etc) that need to be inovked18:52
SergeyLukjanovin fact, node group = node template + number of nodes18:52
jmaronand cluster template = list of node groups?18:53
SergeyLukjanovlist of node templates18:53
jmaronah18:53
jspeidelok.  I assume the plugin will supply the available components that could be included in a node_template?18:53
jmaronthough I suppose the template could provide a "default value" as well, but I don't want to go there right now ;)18:53
SergeyLukjanovso, the user can just select the cluster template and specify number of nodes and cluster will be created18:54
SergeyLukjanovjmaron, yep, see it18:54
jspeidela proper use case describing how a user creates these template would be great18:55
jmaronand the interface implies that a user can add a node group beyond the node groups defined in the template18:55
ruheright, it is noted in our mockup18:55
*** aababilov has joined #openstack-meeting-alt18:56
SergeyLukjanov1 node template describes only 1 node group, but you can just add more node templates to cluster18:56
*** malini has joined #openstack-meeting-alt18:56
*** malini has quit IRC18:57
SergeyLukjanovI think that we should go to the #savanna channel to continue discussions due to the end of the time slot18:57
*** malini has joined #openstack-meeting-alt18:57
SergeyLukjanov#endmeeting18:57
*** flaper87 has quit IRC18:58
*** flaper87 has joined #openstack-meeting-alt18:58
SergeyLukjanov[22:31:22]  openstack (~openstack@openstack/openstack) left IRC. (Remote host closed the connection)18:59
*** Nadya has left #openstack-meeting-alt18:59
SergeyLukjanovmeetbot leaves us :(18:59
flaper87SergeyLukjanov: try again18:59
flaper87stupid meetbot18:59
SergeyLukjanov#endmeeting18:59
flaper87openstack: dude, do your job19:00
aignatov2xD19:00
flaper87gosh19:00
flaper87let me try starting our meeting19:00
flaper87#startmeeting marconi19:00
openstackMeeting started Thu May 16 19:00:45 2013 UTC.  The chair is flaper87. Information about MeetBot at http://wiki.debian.org/MeetBot.19:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:00
*** openstack changes topic to " (Meeting topic: marconi)"19:00
openstackThe meeting name has been set to 'marconi'19:00
flaper87w0000000t19:00
flaper87marconi folks around?19:01
maliniyes Sir19:01
* flaper87 wonders what's alessio's nick19:01
aababilovHi! It's Alessio!19:01
flaper87aababilov: there you are19:01
aababilovsi! :)19:01
*** jmaron has left #openstack-meeting-alt19:02
flaper87soooo, it looks like it'll be the three of us today19:02
flaper87https://wiki.openstack.org/wiki/Meetings/Marconi19:02
flaper87that's the schedule for today19:02
flaper87before we get there, is there anything we should know share or something?19:02
*** sarob has joined #openstack-meeting-alt19:02
flaper87kk19:03
*** kgriffs has joined #openstack-meeting-alt19:03
flaper87#topic blueprints19:03
*** openstack changes topic to "blueprints (Meeting topic: marconi)"19:03
flaper87#link https://blueprints.launchpad.net/marconi19:03
*** wirehead_ has joined #openstack-meeting-alt19:03
flaper87I know last week you guys went through the blueprints, I think we should do that again today and see which of those are really esential19:03
*** ametts has joined #openstack-meeting-alt19:03
kgriffskk19:03
flaper87so, we've got the base blueprints for transports and sotrages which are obviously essential19:04
kgriffsI guess we can move grizzly-debt down a notch19:04
kgriffs(or two)19:04
flaper87and we've also got the transport wsgi and mongodb that are essential as well19:04
flaper87kgriffs: +119:04
flaper87#link https://blueprints.launchpad.net/marconi/+spec/grizzly-debt19:04
flaper87kgriffs: done19:04
flaper87#agreed setting grizzly-dept to medium19:05
flaper87#link https://blueprints.launchpad.net/marconi/+spec/config-module19:05
flaper87I think this one is ready19:05
flaper87what do you guys think?19:06
kgriffsyeah, it's been stable for a while now19:06
*** highlycaffeinate has joined #openstack-meeting-alt19:06
flaper87cool, so, I guess we should set it as implemented19:06
kgriffsBTW, I'm thinking to move grizzly-debt to H219:06
*** mattf has quit IRC19:06
flaper87kgriffs: makes sense19:06
* kgriffs does that19:07
flaper87kgriffs: what about adding work items to it?19:07
flaper87like Pay dept on transport19:07
flaper87Pay debt on storage19:07
flaper87and so on19:07
*** highlycaffeinate has left #openstack-meeting-alt19:07
flaper87so we can have a single blueprint and a more organized tasks for that BP19:07
*** ruhe has left #openstack-meeting-alt19:07
kgriffsI like that idea. We haven't been using that feature yet.19:07
kgriffs+119:08
aababilov+119:09
flaper87cool19:09
aababilovI really wondered what exactly means " fix all the broken windows"19:09
flaper87#agreed add work items to grizzly-dept19:09
aababilovdo we have appropriate FIXMEs?19:09
* flaper87 set grizzly-debt as implemented by mistake.19:10
flaper87I put it back as started19:10
* kgriffs seeded with a few work items19:10
flaper87awesome19:10
aababilovok, 'cause now I see just one FIXME and 7 TODOs19:10
flaper87so, I think essential blueprints are set19:11
flaper87aababilov: mmh, perhaps your seeing bugs instead of blueprints ?19:11
flaper87aababilov: https://blueprints.launchpad.net/marconi19:11
aababilovno, I definitely look at https://blueprints.launchpad.net/marconi/+spec/grizzly-debt19:12
flaper87we have qa-clusters set for havana-1 and marked as High19:12
flaper87aababilov: ah sorry, I misunderstood19:12
maliniwe need those set up for the performance testing19:12
*** amyt has quit IRC19:12
flaper87malini: does that have to happen for H-1 ?19:12
*** amyt has joined #openstack-meeting-alt19:12
flaper87H-1 ends by the end of this month, AFAIK19:13
flaper87we've to push that back19:13
kgriffsright, H1 ends on 30th19:13
maliniok..we might take longer to get the salt scripts etc. done for tht19:13
flaper87ok, pushed it back to H-219:14
flaper87we'll revisit it later19:14
aababilovthere is no ref to Gerrit on https://blueprints.launchpad.net/marconi/+spec/qa-cluster but progress is good19:14
flaper87FYI, this is the list of H-1 bps https://launchpad.net/marconi/+milestone/havana-119:14
flaper87aababilov: gooood point19:14
aababilovhttps://review.openstack.org/: Service Temporarily Unavailable19:14
kgriffsflaper87: oz_akan is working on that, may have it done in a week or two19:15
malinitht one is just setting up the servers etc. So do you expect anything in gerrit for tht ?19:15
flaper87kgriffs: qa-clusters you mean?19:15
kgriffsright19:15
flaper87right, I see him there, so, I guess we better wait for his update19:15
kgriffsoz_akan said he couldn't make it to the meeting, will email an update19:16
flaper87sounds good19:16
flaper87next: https://blueprints.launchpad.net/marconi/+spec/service-hooks19:16
kgriffsI think we can keep it in h1 for now - malini do you know if he still plans on finishing this in the next week or two?19:17
malinihe has plans to finish it soon19:17
kgriffsexcellent19:17
flaper87#action oz_akan to send update about the progress on qa-cluster19:17
flaper87:D19:17
kgriffsflaper87: re service-hooks I bumped up that priority because it's required to implement input validation19:17
*** aignatov2 has left #openstack-meeting-alt19:18
kgriffsbut we can postpone to h2. Methinks fixing bugs and closing functionality gaps is more important19:18
*** akuznetsov has quit IRC19:18
flaper87kgriffs: +119:18
kgriffs(i.e., postpone both service-hooks and input-validation)19:18
*** akuznetsov has joined #openstack-meeting-alt19:19
flaper87ok, doing it now if there are no objections19:19
*** akuznetsov has quit IRC19:19
flaper87#info launchpad is REALLY STUPID pushes back a parent and not its dependency (not even a small note)19:19
flaper87next19:20
flaper87I guess the same applies for this one https://blueprints.launchpad.net/marconi/+spec/message-pagination19:20
aababilovwhat is the pagination model?19:21
aababilovwhat has client to specify: the market or the offset?19:21
aababilovmarkeR19:22
flaper87aababilov: the idea is to have a way to paginate messages and queues through the API but it has to be fast and simple for other transports as well19:22
flaper87so far we don't have pagination so we decided to put it in a separated bp19:22
*** IlyaE has quit IRC19:22
*** cp16net is now known as cp16net|away19:22
flaper87kgriffs: news about this one? https://blueprints.launchpad.net/marconi/+spec/v1-obvious-optimizations19:23
flaper87and the pagination one?19:23
flaper87(a couple of more minutes for blueprints and then we'll move on to other topics)19:23
flaper87I think the later can stay for H-119:24
flaper87I think we're in good shape for H-1 https://launchpad.net/marconi/+milestone/havana-119:25
* flaper87 is basically talking to himself :P19:25
flaper87ooooooooooooooooooke-doke19:26
flaper87moving on19:26
flaper87#topic marconiclient adopting oslo's apiclient19:27
*** openstack changes topic to "marconiclient adopting oslo's apiclient (Meeting topic: marconi)"19:27
flaper87so, I like the idea in general19:27
flaper87and having a common client lib throughout openstack makes sense to me19:27
flaper87(fucking gerrit is down)19:28
flaper87aababilov: so, a couple of thoughts about your patch19:28
*** amyt has quit IRC19:28
aababilovbut I have comments in my email19:28
kgriffssorry, got distracted.19:28
* kgriffs is reading log19:28
*** amyt has joined #openstack-meeting-alt19:28
flaper871) I'd like to move that outside openstack/common and put it in common/19:28
aababilovshould I say a couple of words about the library's purpose?19:28
flaper87the reason is that openstack/common belongs  to oslo19:29
flaper87and as for now, we're implementing it as part of marconiclient19:29
flaper87aababilov: go ahead19:29
flaper87kgriffs: aababilov is the guy who has been working on the common client library for openstack19:29
aababilovapiclient contains code that's common for python-*client19:30
kgriffsflaper87: re pagination, we have the idea of marker, limit defined. we just have to solve the fifo+guaranteed delivery19:30
kgriffsaababilov: excellent. nice progress on that19:30
aababilov1) HttpClient (that reissues authentication request for expired tokens)19:30
aababilov2) pluggable authentication:19:31
aababilova) keystone;19:31
flaper87kgriffs: right, I saw it is marked as Good Progress, should it stay in H-1?19:31
aababilovb) endpoint + token19:31
aababilovc) nova legacy19:31
aababilovd) whatever19:31
kgriffsflaper87: yeah, I plan on tackling that in the very near future19:31
flaper87kgriffs: ++19:31
aababilov3) rich exception hierarchy19:31
aababilov4) Manager and Resource base classes19:31
flaper87aababilov: all that sounds good to me19:32
aababilov5) utils for building CLI tools (temporarily)19:32
aababilovwhat do we have now?19:32
kgriffsdoes it do async requests?19:32
flaper87kgriffs: so, the idea would be to use marconiclient as a "test" environment for the api. I think both the api and marconiclient can benefit from this interaction19:33
flaper87and early adopting stage19:33
flaper87kgriffs: good question19:33
aababilovno. Could you show me a sample implementation?19:33
aababilov(implementation of async req)19:33
aababilovok, I'll proceed19:33
kgriffswe were playing around with that in the python-marconiclient prototype19:33
aababilovgerrit is down, but believe me19:34
kgriffshttps://github.com/painterjd/python-marconiclient19:34
aababilovI have written updates for almost all python*clients, except of quantum and swift19:34
aababilovall unit tests pass19:34
flaper87#link https://github.com/kennethreitz/grequests19:34
aababilovCLI utilities work and are backwards-compatible19:34
flaper87aababilov: kgriffs perhaps based on that ^19:34
kgriffsthe reason I ask, is that 1) a event queue client naturally lends itself well to an event-driven model19:35
aababilovthanks!19:35
kgriffsand 2 ) to get your thoughts on eventlet vs greenlet vs that-py3-thing19:35
kgriffs(AKA tulip)19:35
* flaper87 thinks tulip is cool19:35
aababilovgrequests look nice19:36
kgriffsre grequests, that would be cool19:36
flaper87:)19:36
* flaper87 wants pop-tarts19:36
wirehead_My vague sense at the moment is that everybody seems to be a bit gung ho to play with Tulip19:36
kgriffsmaybe we contribute and make it magically work with tulip and py319:36
* kgriffs gives flaper87 a chocolate pop-tart19:37
flaper87w00000t19:37
wirehead_And that's across different OpenStack teams.19:37
kgriffsthere was a discussion at the summit about migrating away from eventlet19:37
aababilovbut my point is to accept apiclient in some form and to begin improving it19:37
kgriffs(the other reason I bring this up)19:38
kgriffsyes19:38
flaper87aababilov: that's the plan19:38
kgriffsasync can come later19:38
kgriffsfeel free to use marconi as a guinea pig19:38
flaper87kgriffs: there was a question in the review about whether to have the client under openstack/common/apiclient or common/apiclient19:38
aababilovcould we accept apiclient to oslo? it's an incubator, isn't it?19:38
kgriffs(for apiclient in general)19:38
aababilovmarconiclient will be the first user19:38
flaper87my suggestion is to put it in common19:38
kgriffsflaper87: kk19:39
flaper87because openstack/common/ is oslo's holy ground19:39
flaper87and dude, you don't want to mess with it19:39
flaper87aababilov: yep, it's an incubator lib BUT, it is always good to have the base API sorted out before letting anything land there19:39
flaper87so, first give it a try somewhere19:39
flaper87and then move it there19:40
kgriffsoic19:40
flaper87so that people *know* how that thing should work19:40
kgriffsheh19:40
aababilovsure, but what will be the criterions to put it to oslo?19:40
aababilovmarconiclient lacks unit tests19:40
* kgriffs is finally caught up with the conversation19:40
flaper87aababilov: yep, it lacks of everything19:40
aababilovnovaclient and keystoneclient have them19:40
aababilovwe have integration tests in tempest19:41
aababilovI could update horizon19:41
flaper87aababilov: yep but we need many unittests 1) for apiclient (which are the ones that will be ported to oslo) and 2) for marconiclient19:41
*** highlycaffeinate has joined #openstack-meeting-alt19:41
kgriffshow quickly could you get apiclient accepted for one of the mature projects?19:42
*** highlycaffeinate has left #openstack-meeting-alt19:42
aababilovit depends on maintainers19:42
flaper87TBH, I wouldn't do that19:42
aababilovit took 3-4 days to update all clients19:42
flaper87I'd rather test it and improve it on marconi than going out there and implement it in mature clients19:43
kgriffs+119:43
kgriffsi was just thinking it may be a real chore to get it accepted into those other projects19:43
kgriffs(without any track record)19:43
flaper87exactly19:44
flaper87so, we've got 15 mins left and I'd love to share some thoughts about zmq transport19:44
aababilovwell, apiclient is not written from scratch19:44
aababilovit's mostly an aggregation of code that already resides in *clients19:44
flaper87aababilov: yeah, that's cool, I mean, it's based on other clients sweat19:44
flaper87aababilov: TBH, that last sentence scares a bit19:45
flaper87code from different places put in the same package...19:45
flaper87it's not bad but it definitely neeeds to be tested a LOT19:45
flaper87that's my thinking19:45
flaper87so, again, I'd suggest you, if you agree, to use marconiclient as test environment19:46
flaper87and both projects will ebnefit from each other19:46
flaper87you can always go ahead and propose it to other clients as well19:46
flaper87but if something changes in 1 client, you'll have to update all of them19:46
aababilovsure, I agree to use marconiclient!19:46
flaper87cooooooooool19:46
flaper87aababilov: feel free to join #openstack-marconi :)19:47
aababilovdone :)19:47
flaper87any other thoughts here ?19:47
flaper87cool19:47
flaper87moving on19:47
flaper87#topic zmq transport and flaper87 crazy ideas19:48
*** openstack changes topic to "zmq transport and flaper87 crazy ideas (Meeting topic: marconi)"19:48
flaper87#link https://etherpad.openstack.org/marconi-zmq19:48
flaper87so, I was working on that ^19:48
flaper87which seems incomplete19:48
flaper87and that's what it actually is19:48
flaper87and then thought that, if we've to implement an RPC like API for the zmq stuff, why don't we use the same rpc like protocol for all the transports?19:49
* flaper87 runs away19:49
* flaper87 runs fast, fast faaaaaaaaaaaaaaast19:49
kgriffsheh19:49
flaper87the idea would be to use the same "RPC" like for HTTP communications as well19:49
flaper87benefits:19:49
flaper871) same code19:49
flaper872) same verifications19:50
flaper873) less code to maintain19:50
flaper87drawbacks:19:50
flaper871) RPC over HTTP, erm, that sounds like a bunch of POST requests19:50
flaper872) it's all json based and more data will flight to / from the server19:51
kgriffsRPC over HTTP is fuuuuuugly19:51
flaper87other benefit I see is that the same protocol can be used for websocket19:51
flaper87so, that's the crazy idea19:51
kgriffswebsocket is ok19:51
flaper87if it really sucks I'll just STFU19:51
kgriffswell, we have a couple paradigms19:52
aababilovgerrit is alive!19:52
kgriffsone is RPC, one is REST19:52
flaper87yeah19:52
kgriffsjust thinking out loud. so...19:52
flaper87go ahead19:52
flaper87you've 8 mins before you'll have to shut your brain down19:53
kgriffsit would be interesting to layer HTTP and RPC over the controllers19:53
kgriffsjust means the controllers have to support the combined set of everything REST and RPC need19:53
kgriffs:p19:53
flaper87#link https://github.com/openstack/glance/blob/master/glance/common/rpc.py19:53
flaper87kgriffs: there you can have an idea of how that could work ^19:54
kgriffskk19:54
kgriffsI'll have to noodle on this19:54
flaper87so, I'd say, lets think this a bit more and elaborate the idea a bit better19:54
flaper87#action flaper87 elaborate RPC idea better and get more pros / cons19:55
flaper87I'll work on the RPC spec anyway because we'll need that for zmq19:55
kgriffsfor now, I vote keeping the HTTP transport RESTful, but I'm open to doing something oslo-ish with the other transports19:55
flaper87kgriffs: cool19:55
flaper87so, speaking of oslo19:55
flaper87kgriffs: did you read the pad?19:56
flaper87there are some benefits from using it19:56
* flaper87 just realized he jumped the tests status update19:56
flaper87malini: SO SO Sorry19:56
*** jbresnah has joined #openstack-meeting-alt19:56
flaper87how much time you need for that?19:57
malinijust a couple of min19:57
flaper87ETOOMANY TOPICS19:57
malinibut finish up the zmq stuff first19:57
flaper87malini: not much to say for now, just wanted to share those 2 was for doing it so we can start thinking about that19:57
flaper87plus, I wanted to tell the crazy idea19:58
flaper87:P19:58
flaper87moving on19:58
kgriffsflaper87: I saw the pad, will do some ponderin'19:58
flaper87kgriffs: cool, thanks a lot19:58
flaper87I'll share with cppcabrera as well19:58
flaper87#topic system tests status update19:58
*** openstack changes topic to "system tests status update (Meeting topic: marconi)"19:58
flaper87malini: go ahead19:58
maliniI have fixed the large bulk of the flake8 errors in the system tests. The remaining few are module import errors.   That one looks a little tricky & we are still figuring out how to fix tht.19:58
flaper872 mins19:58
kgriffsdoes it look like a flake8 bug?19:59
* kgriffs famous last words19:59
flaper87malini: cool, did you try asking mordred as well? (Monty)19:59
mordredaroo?19:59
flaper87he implemented that for most projects19:59
malinisure..19:59
flaper87mordred: there you are :D19:59
mordredsup19:59
flaper87malini: ^19:59
maliniI am having trouble getting some of my modules mergeed19:59
*** sarob_ has joined #openstack-meeting-alt20:00
maliniflake8 claims its not a module20:00
flaper87we've just 1 min left20:00
mordredmalini: can you point me towards a failing review?20:00
malinisure.. will ping u offline..we are almost out of time here20:00
flaper87guys, I've got to end the meeting now! malini thanks for your hard work on tests. ++ for you20:00
malinithanks :)20:01
flaper87Thanks guys, really great meeting full of interesting topics20:01
flaper87#endmeeting20:01
*** openstack changes topic to " (Meeting topic: savanna)"20:01
openstackMeeting ended Thu May 16 20:01:23 2013 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)20:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-05-16-19.00.html20:01
kgriffscheers20:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-05-16-19.00.txt20:01
openstackLog:            http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-05-16-19.00.log.html20:01
*** esheffield has joined #openstack-meeting-alt20:01
markwashglance meeting soon20:01
*** aababilov has left #openstack-meeting-alt20:02
*** qwerty_nor has quit IRC20:02
*** sarob__ has joined #openstack-meeting-alt20:02
*** ashwini has joined #openstack-meeting-alt20:02
flaper87\o/20:02
flaper87\o/20:02
flaper87\o/20:02
markwashshall we?20:02
flaper87yep20:02
Guest25612hi20:02
ameademay have to leave early so lets go20:02
*** sarob has quit IRC20:02
markwash#startmeeting glance20:03
Guest25612+120:03
openstackMeeting started Thu May 16 20:03:01 2013 UTC.  The chair is markwash. Information about MeetBot at http://wiki.debian.org/MeetBot.20:03
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.20:03
flaper87yep20:03
*** openstack changes topic to " (Meeting topic: glance)"20:03
flaper87\o/20:03
openstackThe meeting name has been set to 'glance'20:03
iccha_hey20:03
flaper87\o/20:03
markwashhi errbody20:03
flaper87\o/20:03
*** Guest25612 is now known as nikhil__20:03
flaper87that's a body execption20:03
flaper87exception*20:03
jbresnahwave20:03
markwashI was out for most of last week20:03
*** sarob_ has quit IRC20:03
markwashtoday, i don't have a really set agenda (not one that I recall, at least!)20:03
*** lindj_ has joined #openstack-meeting-alt20:04
jbresnahmarkwash: async workers is an outstanding topic20:04
*** ErikB has quit IRC20:04
* flaper87 raises both hands \o/20:04
markwashmy main item is to target as many blueprints in https://blueprints.launchpad.net/glance/havana20:04
jbresnahif there is time for a discussion that may be a good topic20:04
nikhil__I may have a dependency level20:04
esheffieldthat's one I'm particularly interested in20:04
nikhil__and def asyn workers is most imp20:04
markwashokay, cool20:04
esheffieldasync workers that is20:04
markwashfirst, does anybody have announcements?20:04
*** qwerty_nor has joined #openstack-meeting-alt20:05
nikhil__who's getting married?20:05
flaper87any new kids ?20:05
markwashI noticed the new xfer service work was started20:05
jbresnahmarkwash: should better test suite bp be targeted for havana?20:05
jbresnahflaper87: actually....20:05
jbresnahflaper87: my wife is due with a new kid in < 4 weeks20:06
iccha_thats awesome jbresnah20:06
jbresnahso if i seem panicky lately....20:06
*** ErikB has joined #openstack-meeting-alt20:06
jbresnahalso i may disaapear for a few weeks at that point20:06
jbresnahjust fyi20:06
markwashcool20:06
*** cp16net|away is now known as cp16net20:06
iccha_so far here are the items i see mentioned 1. async workers - updates 2. transfer service 3. test suite20:06
flaper87jbresnah: congrats ;)20:07
markwashI guess lets quickly see if we can make any progress on blueprint targeting, and then discuss async workers?20:07
nikhil__jbresnah: gud luk20:07
ameademarkwash: +120:07
flaper87markwash: +120:07
iccha_+120:07
markwash#topic blueprint targeting20:07
*** openstack changes topic to "blueprint targeting (Meeting topic: glance)"20:07
nikhil__+120:07
markwash#link https://blueprints.launchpad.net/glance/havana20:07
*** jrodom has quit IRC20:07
markwasheverything there that is not targeted should be20:07
markwashand some of the ones with targets might be wrong, considering h1 is not very far away20:08
nikhil__markwash: cloning u mean?20:08
nikhil__I heard a rumour that it's not in havana?20:08
iccha_when is h1?20:09
markwashMay 30th20:09
markwash2 weeks from today20:09
*** malini has quit IRC20:09
rosmaita_that is pretty soon20:09
jbresnahyeah that is soon20:09
nikhil__soon-=120:09
markwashnikhil__: I don't know what you mean about cloning20:10
markwashits in the havana list, that's all I know20:10
nikhil__markwash: ohk, just wanted to make sure20:10
iccha_markwash: is bcwaldon working on multiple locations?20:10
markwashlooking over the list, it seems to me that we still need to sort out upload-download workflow20:10
markwashbut maybe that is blocked on our async workers discussion?20:10
markwashiccha_: not that I've noticed20:11
jbresnahmarkwash: i think that is right20:11
nikhil__markwash: yes20:11
jbresnahmarkwash: i think up/down is largely sorted out based on the fallout of async20:11
iccha_markwash: it is a dependency but we can figure out the workflow wihtout it assuming async workers will work20:11
iccha_do we want to schedule a time to talk about async workers in openstack-glance sometime? is there an intial layout?20:12
flaper87I wrote down some ideas and considerations20:12
iccha_i am sorry i missed prev 2 meetings not sure if it was already done20:12
flaper87and I wanted to share that today if possible20:12
markwashsounds good20:12
rosmaita_please share!20:12
flaper87but I'm ok with doing it in a separate meeting if time is not enough20:12
markwashI guess we're not going to make a ton of progress with bp targeting without sorting out the async stuff20:13
markwashso let's move on to that soon, but first20:13
flaper87rosmaita_: that will cost a couple fo pop-tarts20:13
markwashis anyone else interested in tackling multiple image locations?20:13
rosmaita_dafuq?20:13
flaper87markwash: if no one is going to work on that20:13
flaper87I can take it20:13
jbresnahmarkwash: i could do it20:13
flaper87or he ^20:14
flaper87:D20:14
jbresnahflaper87: or we could tag team it...20:14
flaper87jbresnah: +120:14
flaper87sounds like a plan20:14
jbresnahmarkwash: i am pretty interested in seeing it get done, so if it is stalled out i could get on it20:14
markwashcool, I'll assign jbresnah then20:14
rosmaita_with your timezone separation, i think you can work on it continuously20:14
markwashhaha20:14
jbresnahheh20:14
flaper87LOL20:14
flaper8724h20:14
flaper87:P20:14
*** sarob__ has quit IRC20:15
jbresnahmarkwash: should we make sure that is cool with bcwaldon first?20:15
markwashmeh20:15
* jbresnah doesn't like to step on toes20:15
markwashI'll talk to him about it tomorrow at some point20:15
jbresnahok20:15
markwashdon't worry in this case20:15
markwashokay, anything else before we launch into async workers?20:15
* flaper87 is worried about https://blueprints.launchpad.net/python-glanceclient/+spec/glance-client-v220:16
iccha_11 am EST monday in #openstack-glance we will be discussing protected properties in anyone is interested in joining in20:16
flaper87iccha_: awesome20:16
markwashflaper87: ah good point20:16
iccha_https://etherpad.openstack.org/public-glance-protected-props is link iwth some notes20:16
markwashflaper87: once we get that, we can look at a major version bump in glanceclient as well20:16
markwashwhich is good for dropping legacy20:17
flaper87markwash: exactly20:17
flaper87so, I see there hasn't been any movement there20:17
jbresnahiccha: i will try to make that, a little bit early for me but possible20:17
markwashhonestly, I'm kind of out of touch with glance client, and I'd love if there could be a sort of glanceclient lieutenant to help usher us through the process20:17
markwashand manage releases20:17
flaper87I can help with that20:17
markwashwell, I'd like to be consulted about managing releases, of course20:18
markwash:-)20:18
flaper87sure, as for now, I'm worried about the API v2 not being fully supported20:18
flaper87and people asking for it T_T20:18
flaper87:P20:18
markwashcool, flaper87 let's touch base offline to see how we can move forward20:18
flaper87markwash: +120:18
rosmaita_i do know that v2 image sharing is in glance client20:19
iccha_finally got merged after 2+ months :p20:19
flaper87iccha_: :D20:19
* markwash is sad about dropping the ball on that merge20:19
markwashsorry to drag you through all those "restores"20:20
nikhil__async workers pls?20:20
markwash#topic async workers20:20
*** openstack changes topic to "async workers (Meeting topic: glance)"20:20
* markwash passes the mic to flaper87 20:20
* flaper87 clears his voice20:21
flaper87so, these are some of the thoughts I had related to async workers https://etherpad.openstack.org/glance-async-worker20:21
flaper87I took more time to think about what could happen / should be changed if that happens20:21
flaper87and how can that be implemented20:21
nikhil__one comment20:22
nikhil__inline20:22
jbresnahi am not sure there should be 1 way to implement the async worker...20:22
markwashlooking at the two ideas for implementation (1: green threads in the api, 2: async workers across rpc), could we make 1 vs. 2 a configuration setting?20:22
flaper87as shown in the pad, the 2 basic ideas are: 1) KISS and do it within glance-api 2) Use something more "complex" and keep it in a separate service (with scheduler and that kind of things)20:22
jbresnahi would like to see a plugin interface20:23
jbresnahto glance-api proper it may always look like #120:23
jbresnahbut the plugin may implement with #220:23
esheffieldto me it feels like this may be a wheel that's getting reinvented in several places in openstack20:23
markwashjbresnah: +1 that makes sense to me a lot20:23
flaper87jbresnah: markwash indeed, that could be addressed with the new oslo's messaging api20:23
flaper87" For both ideas it would be nice to use the new messaging "20:24
flaper87because it supports blocking executors and eventlet based executors that could be used through an in-memory queue20:24
jbresnahi can see cases where messaging may not be needed20:24
flaper87jbresnah: agreed20:24
nikhil__can we discuss based on use cases20:25
*** IlyaE has joined #openstack-meeting-alt20:25
nikhil__this looks like a wild goose chase20:25
jbresnahnikhil: good plan20:25
markwashwell, one big use case is devstack20:25
*** dmitryme has quit IRC20:25
markwashto me that's the driving use case for in-api green threads / eventlet20:26
iccha_could u elaborate on the devstack use please markwash ?20:26
flaper87markwash: ^20:26
markwashwell, whatever we want to do, we want it to work in super simple deployments like devstack20:27
markwashand it would be nice to have those working without adding a generally unnecessary extra process to track20:27
iccha_which makes the case for making it configurable stronger20:27
jbresnahwhat is something that devstack would use async workers for?20:27
nikhil__QE20:28
nikhil__is what comes to mind20:28
markwashimport and export I guess20:28
markwashQE?20:28
flaper87markwash: devstack is already capable of configuring the oslo's rpc implementation20:28
jbresnahumm, sorry20:28
flaper87I think import / export is one of the main points20:28
nikhil__do we really need a export?20:28
flaper87followed by image introspection20:28
nikhil__I feel like totally out of sync then20:28
jbresnahyeah, i meant how would async be used as a use case20:28
nikhil__jbresnah: yeah20:29
*** kgriffs has left #openstack-meeting-alt20:29
nikhil__i mean : me too20:29
jbresnahnot what glance use cases do we want to not disrupt20:29
markwashwe're talking about different api calls that process something in the background20:29
flaper87exactly20:29
markwashso devstack would have the same use cases20:30
markwashas any other deployment20:30
flaper87I see that being useful in different scenarios (like those in the pad)20:30
jbresnahnod20:30
markwashjust with the additional constraint of it being all in one box20:30
jbresnahi just think it may be helpful to walk through how an example async worker would work20:30
nikhil__I interpreted jbresnah's question as "is there a need to have it in devstack"?20:30
markwashthere is also the usecase where an existing deployment upgrades and doesn't want to change their deployment layout at first20:30
jbresnahfor example, an async worker that handles the upload20:30
jbresnahor an async worker that handles image conversion20:31
iccha_markwash: are u driving towards why we need both types of implementation and configurable?20:31
markwashiccha_: yes I think so20:31
markwashthat probably wasn't a clear as I should have made it :-)20:31
markwashs/a clear/as clear/20:31
nikhil__agree20:32
markwashdevstack and existing deployments might want the new asynchronous operation features without changing their deployment layout (ie, not adding another process / moving part)20:32
nikhil__however, for even a small production level deployment20:32
flaper87I agree on the fact that both methods should be supported20:32
flaper87we can vote :)20:32
nikhil__do we think that threads on api nodes could handle just fine?20:33
jbresnahi would think that both methods would be supported by way of what ever plugin was in use20:33
nikhil__my concern is20:33
nikhil__if we are waiting for both support20:33
jbresnahi think glance would dictate a plugin interface20:33
flaper87nikhil__: I bet they would, my point is, there's already some work going on on similar code in oslo, so, why should we write it again?20:33
jbresnahbut allow the plugins to implement that iface in wahtever way it felt best20:33
markwashflaper87: ohh20:34
nikhil__and if just async workers on api nodes do not work, then effort is futile to a great extent20:34
nikhil__flaper87: gotcha20:34
iccha_does doing a async worker separately and not as a thread in api , does that necessarily mean that it becomes heavier and complex?20:34
markwashflaper87: from glance's perspective, the async processing should just be an object or group of objects20:34
flaper87so, this is the current state in oslo's new messaging (RPC) api https://github.com/markmc/oslo-incubator/tree/messaging/openstack/common/messaging20:34
iccha_could it be configurable to make the worker light weight with say one thread vs more complex with many threads or something like that20:35
markwashif messaging supports making the choice of local or remote really easy, we should use that20:35
flaper87markwash: I think it does, I'll work on a POC for that20:35
flaper87#action flaper87 work on a POC for using oslo's messaging20:35
jbresnahiccha: i would think that would be up to the worker20:35
markwashwhat I'm saying is that it doesn't make sense for async processing to have a "messaging" interface, rather messaging should be an implementation detail of what's underneath the interface20:36
*** sacharya has quit IRC20:36
nikhil__+120:36
flaper87as explained in the pad, I see some extra benefits moving towards that (aling glance with oslo and updating the notification stuff)20:36
jbresnahiccha: i would like to see glance proper have a lightweight thread to manage/start/check status/etc of all it ... what mark said20:36
markwashI think it makes sense that oslo-common messaging gets us where we want to be20:36
flaper87markwash: In my head I see some higher-level API in glance that wraps oslo's calls20:37
flaper87like, Async.process_this_stuff(...)20:37
markwashI guess the POC is the right first step in any case, though20:37
flaper87markwash: yep20:37
markwashwe can take a look at it, and offer an alternative if one seems necessary20:37
flaper87sounds like a plan20:37
flaper87I'm not saying that will work :P20:37
flaper87if you don't see me on-line20:38
flaper87I'm ok, I just ran away20:38
flaper87:D20:38
markwashdo we need task tracking with this approach?20:39
flaper87as a sidenote, I didn't work on any POC because I first wanted to agree with everyone on some usecases and perhaps a flow20:39
* markwash thinks we should ignore his task tracking comment for now20:39
jbresnahi am not sure what was decided20:39
*** yidclare1 has joined #openstack-meeting-alt20:39
flaper87jbresnah: flaper87 work on a POC for using oslo's messaging20:40
*** ashwini has quit IRC20:40
nikhil__does marconi have influence on that?20:41
nikhil__flaper87: ^^20:41
flaper87nikhil__: not sure I understand20:42
jbresnahi dont understand why messaging is a first class design choice yet20:42
nikhil__https://wiki.openstack.org/wiki/Marconi20:42
nikhil__flaper87: ^^20:42
jbresnahi would like to be free to have a custom plugin that does its work via fork20:42
jbresnahor via a thread20:42
*** yidclare has quit IRC20:42
jbresnahetc20:42
jbresnahi dont like forcing messaging20:42
nikhil__+120:42
flaper87jbresnah: well, that's basically what oslo's messaging does20:42
jbresnahit seems like the wrong angle of attack20:42
iccha_could we have like a pro cons list in the etherpad20:42
* markwash agrees20:43
flaper87but with a simple queue that can be either in memory or through MB20:43
jbresnahflapper: that is about messaging tho20:43
flaper87etherpad stopped working here20:43
markwashflaper87: but, there are some cases where you wouldn't use messaging at all, like if you say used fabric to ssh to another node to initiate processing20:43
jbresnahseems like the topic has the wrong focus20:43
nikhil__last I saw this, we'r trying to do store to store trnasfers using a asycn worker20:43
markwashflaper87: or if you wanted to kick off some processing on a node that is really "close" to the data, like a zerovm or docker worker20:44
jbresnahright20:44
flaper87mmhh20:44
jbresnahi think the first bit of work is: define a async worker worlflow20:44
jbresnahdefine the state machine from glance api perpective and client persoective20:44
jbresnahthen define an interface that async workers will implement20:45
flaper87yeap, that's what I was expecting from this meeting20:45
jbresnahafter that implementation details will make more sense20:45
*** bdpayne_ has joined #openstack-meeting-alt20:45
flaper87so we could start working on some POC after that20:45
jbresnahoh, and i think task #0 should be a requirements definition20:46
jbresnahso we know what we are solving exactly20:46
jbresnahi think parts of those things exist20:46
nikhil__jbresnah: perfect!20:46
iccha_+120:46
iccha_because i was starting to get a little lost here with all the different ideas20:46
nikhil__:)20:46
iccha_requirements, with a pro con list of basic approaches20:47
markwashI'd like to see what "import" would look like in the v2 code, using async processing20:47
jbresnahwell...20:47
jbresnahi am not sure approaches whould be married with requirements20:47
nikhil__4 mins till open discussion? (if someone has one - just a friendly reminder)20:47
jbresnahrequirements with a list of pros and cons for the requirement20:47
jbresnahbut it would be good to agree on the requirements up front without mixing in impl discussion20:47
jbresnahmarkwash: i think that is the perfect use case to define around20:48
flaper87+120:48
iccha_+120:48
markwashI might not be good at "requirements" though exactly, more like "here's what I want the interface to look like"20:48
rosmaita_+120:48
rosmaita_(that was to jbreshah, not markwash!)20:49
markwash#action markwash try to code up a stubbed version of import in v2, maybe it will help motivate requirements20:49
nikhil__:D20:49
nikhil__agree on the "import"20:49
nikhil__that's the most imp requirement in my mind20:50
* markwash is trying to "first do no harm" to this discussion, not sure its working20:50
jbresnahmarkwash: i can help with requirements20:50
*** bdpayne has quit IRC20:50
*** djohnstone has quit IRC20:50
jbresnahtho... i am not sure i am in the position to know what people want exactly...20:50
*** jspeidel has left #openstack-meeting-alt20:51
esheffieldI'd like to help with that as well20:51
nikhil__someone pls create an action item for this20:51
nikhil__requirements gathering20:51
nikhil__that's software engineering 10120:52
flaper87does etherpad work for you?20:52
iccha_in terms of import export there are a few etherpads rosmaita_ created20:52
flaper87it stopped working here20:52
jbresnahetherpad works20:52
flaper87mmh20:52
* nikhil__ feels like sync failed20:52
rosmaita_not here, stopped working long time ago20:52
esheffieldnot worker for me either20:52
jbresnahoh, sorry i thought you meant as a forum for requirement gathering20:52
iccha_that is separate jbresnah :)20:52
jbresnahetherpad has been falky for me this entire houtr20:52
flaper87ok20:52
iccha_the import export stuff is just for us to look at import export work flow and req gathering is for async worker20:53
jbresnahagreed20:53
flaper87so, what are the next steps on this topic?20:54
flaper87markwash: will work on that stubbed version of import in v220:54
flaper87right?20:54
jbresnahsomeone should take point on requirements gathering20:54
nikhil__pros and cons list for me pls20:54
nikhil__i can take requirements gathering20:55
markwashflaper87: right20:55
flaper87#action nikhil__ to work on requirements gathering20:55
nikhil__cool20:55
jbresnahnikhil: cool, keep in the loop please20:55
markwashnikhil__: ditto what jbresnah said for me20:55
iccha_flaper87: have u added ur etherpad to async workers bp?20:55
flaper87nikhil__: and me :D20:55
flaper87iccha_: nope, I created it today because I took those notes offline20:56
flaper87I'll do20:56
nikhil__think I'll just keep all posted on openstack-glance20:56
flaper87nikhil__: please, have mercy with europeans :D20:56
flaper87well, I should definitely sleep more but that's another story20:56
nikhil__you mean the timezone rt?20:56
nikhil__kk20:56
flaper87nikhil__: yep :D20:56
markwashany time left for open discussion?20:56
iccha_3 mins20:57
flaper873 mins20:57
nikhil__I dont have any agenda and need to go in 3 mins20:57
flaper87markwash: what iccha_ said :D20:57
markwash#topic open discussion20:57
*** openstack changes topic to "open discussion (Meeting topic: glance)"20:57
markwashI'd like to have a meeting time that is more friendly to the other hemisphere20:58
markwashat least 1/month20:58
markwashnot sure if I already said that here, or in #openstack-glance20:58
rosmaita_+120:58
flaper87this hemisphere was sleeping when you said that :P20:58
nikhil__15UTC works for me20:58
markwashthere are people who totally can't make it b/c of the time20:58
flaper87markwash: agreed20:59
markwashdoes 1/month sound okay?20:59
flaper87+120:59
markwashwe can do a trial for the first meeting in june, maybe20:59
markwashor next week if the need is more urgent20:59
jbresnahi am meeting time flexible20:59
markwashjbresnah is on a beautiful beach right now anyway21:00
jbresnahheh21:00
flaper87haha21:00
jbresnahits so hard to type with all this sand in my keyboard21:00
markwashhaha21:00
nikhil__is the meeting over?21:01
markwashI guess so21:01
nikhil__i mean in terms of discission?21:01
markwashanything else?21:01
* markwash is a little distracted today, sorry21:01
rosmaita_markwash: did not split the upload/download BP yet21:01
rosmaita_started 2 discussion etherpads first21:01
markwashlinks?21:02
rosmaita_but, they appear to be unavailable at the moment21:02
markwashah21:02
rosmaita_links in  BP21:02
markwashgood, I'll have a look21:02
* nikhil__ says bye21:02
markwashI guess that's all for us today, folks21:02
markwashthanks!21:02
flaper87cool21:02
flaper87thanks21:02
markwash#endmeeting21:03
*** openstack changes topic to " (Meeting topic: savanna)"21:03
openstackMeeting ended Thu May 16 21:03:03 2013 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)21:03
openstackMinutes:        http://eavesdrop.openstack.org/meetings/glance/2013/glance.2013-05-16-20.03.html21:03
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/glance/2013/glance.2013-05-16-20.03.txt21:03
openstackLog:            http://eavesdrop.openstack.org/meetings/glance/2013/glance.2013-05-16-20.03.log.html21:03
*** SergeyLukjanov has quit IRC21:03
*** esheffield has left #openstack-meeting-alt21:03
*** sarob has joined #openstack-meeting-alt21:05
*** ametts has quit IRC21:06
*** jbresnah has left #openstack-meeting-alt21:06
*** dhellmann-away has quit IRC21:08
*** sarob has quit IRC21:10
*** sarob has joined #openstack-meeting-alt21:11
*** p4tux has joined #openstack-meeting-alt21:16
*** flaper87 has left #openstack-meeting-alt21:16
*** p4tux has left #openstack-meeting-alt21:17
*** p4tux has joined #openstack-meeting-alt21:17
*** hartsocks has left #openstack-meeting-alt21:19
*** malini has joined #openstack-meeting-alt21:21
*** gorozco1_ has joined #openstack-meeting-alt21:22
*** p4tux has quit IRC21:22
*** wirehead_ has left #openstack-meeting-alt21:26
*** malini has quit IRC21:26
*** jrodom has joined #openstack-meeting-alt21:27
*** gorozco1_ is now known as p4tux21:45
*** amyt has quit IRC22:14
*** jrodom has quit IRC22:26
*** sarob_ has joined #openstack-meeting-alt22:27
*** sarob__ has joined #openstack-meeting-alt22:28
*** sarob_ has quit IRC22:28
*** yidclare1 has quit IRC22:29
*** sarob has quit IRC22:31
*** yidclare has joined #openstack-meeting-alt22:33
*** rnirmal has quit IRC22:36
*** p4tux has quit IRC23:15
*** p4tux has joined #openstack-meeting-alt23:16
*** ErikB has quit IRC23:19
*** p4tux has quit IRC23:20
*** qwerty_nor has quit IRC23:25
*** lindj_ has quit IRC23:31
*** jcru has quit IRC23:38
*** ErikB has joined #openstack-meeting-alt23:50
*** kagan has joined #openstack-meeting-alt23:51
*** sarob__ has quit IRC23:53
*** sarob has joined #openstack-meeting-alt23:54
*** sarob has quit IRC23:58
*** IlyaE has quit IRC23:59
*** malini has joined #openstack-meeting-alt23:59
*** ErikB has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!