Thursday, 2013-03-21

*** jcru|away is now known as jcru00:11
*** jcru has quit IRC00:27
*** vipul is now known as vipul|away00:40
*** amyt has joined #openstack-meeting-alt00:54
*** amyt has joined #openstack-meeting-alt00:55
*** vipul|away is now known as vipul00:57
*** grapex has quit IRC01:23
*** jcru has joined #openstack-meeting-alt01:25
*** esp has joined #openstack-meeting-alt03:19
*** jcru is now known as jcru|away03:19
*** jcru|away is now known as jcru03:22
*** esp has quit IRC04:06
*** kagan has joined #openstack-meeting-alt04:26
*** esp has joined #openstack-meeting-alt04:36
*** vipul is now known as vipul|away04:38
*** esp has quit IRC04:40
*** vipul|away is now known as vipul04:50
*** amyt has quit IRC04:57
*** jcru is now known as jcru|away05:02
*** amyt has joined #openstack-meeting-alt06:27
*** amyt has quit IRC06:41
*** BobBall has quit IRC07:08
*** kagan has quit IRC07:41
*** johnthetubaguy has joined #openstack-meeting-alt08:26
*** kagan has joined #openstack-meeting-alt08:31
*** jcru|away is now known as jcru09:26
*** kagan has quit IRC09:49
*** jcru has quit IRC11:11
*** kagan has joined #openstack-meeting-alt11:34
*** kagan has quit IRC11:54
*** grapex has joined #openstack-meeting-alt13:45
*** grapex has quit IRC13:47
*** grapex has joined #openstack-meeting-alt13:47
*** cloudchimp has joined #openstack-meeting-alt13:52
*** jcru has joined #openstack-meeting-alt13:58
*** rnirmal has joined #openstack-meeting-alt14:03
*** jcru is now known as jcru|away14:20
*** jcru|away is now known as jcru14:22
*** yidclare has joined #openstack-meeting-alt14:26
*** amyt has joined #openstack-meeting-alt14:30
*** amyt has quit IRC14:32
*** amyt has joined #openstack-meeting-alt14:32
*** jcru is now known as jcru|away14:46
*** cp16net is now known as cp16net|away15:06
*** cp16net|away is now known as cp16net15:07
*** cp16net is now known as cp16net|away15:10
*** cp16net|away is now known as cp16net15:10
*** jcru|away is now known as jcru15:11
*** cp16net is now known as cp16net|away15:18
*** cp16net|away is now known as cp16net15:19
*** imsplitbit has joined #openstack-meeting-alt15:28
*** sacharya has joined #openstack-meeting-alt15:31
*** amyt has quit IRC15:36
*** amyt has joined #openstack-meeting-alt15:37
*** amyt has quit IRC15:39
*** amyt has joined #openstack-meeting-alt15:39
*** jcru is now known as jcru|away15:42
*** jcru|away is now known as jcru15:50
*** yidclare has quit IRC16:04
*** grapex has quit IRC16:05
*** heckj has joined #openstack-meeting-alt16:18
*** grapex has joined #openstack-meeting-alt16:37
*** imsplitbit has quit IRC16:45
*** yidclare has joined #openstack-meeting-alt16:54
*** cp16net is now known as cp16net|away16:56
*** yidclare has quit IRC17:01
*** cp16net|away is now known as cp16net17:02
*** esp has joined #openstack-meeting-alt17:02
*** esp has left #openstack-meeting-alt17:02
*** yidclare has joined #openstack-meeting-alt17:03
*** yidclare has quit IRC17:04
*** yidclare has joined #openstack-meeting-alt17:05
*** cp16net is now known as cp16net|away17:21
*** cp16net|away is now known as cp16net17:23
*** yidclare has quit IRC17:27
*** yidclare has joined #openstack-meeting-alt17:29
*** bdpayne has quit IRC17:30
*** bdpayne has joined #openstack-meeting-alt17:31
*** rmohan has quit IRC17:31
*** rmohan has joined #openstack-meeting-alt17:33
*** rmohan has quit IRC17:34
*** rmohan has joined #openstack-meeting-alt17:35
*** rmohan has quit IRC17:35
*** rmohan has joined #openstack-meeting-alt17:35
*** rmohan has quit IRC17:40
*** rmohan has joined #openstack-meeting-alt17:40
*** rmohan has quit IRC17:40
*** rmohan has joined #openstack-meeting-alt17:41
*** jdprax has joined #openstack-meeting-alt17:52
*** kgriffs has joined #openstack-meeting-alt17:56
*** bryansd-home has joined #openstack-meeting-alt17:56
kgriffsFYI the Marconi meeting will be in one hour (1900 UTC)18:02
kgriffsI'm available for a few minutes to answer any questions re Falcon.18:02
*** treeder has joined #openstack-meeting-alt18:04
kgriffsFYI the Marconi meeting will be in one hour (1900 UTC)18:04
* kgriffs curses daylight saving's time18:04
*** bryansd-home has quit IRC18:06
*** bryansd-home has joined #openstack-meeting-alt18:07
*** bryansd-home has quit IRC18:11
*** bryansd-home has joined #openstack-meeting-alt18:12
heckjkgriffs: stupid question - what's "Marconi"?18:13
*** rmohan has quit IRC18:19
*** rmohan has joined #openstack-meeting-alt18:21
kgriffsheckj: https://wiki.openstack.org/wiki/Marconi18:24
*** johnthetubaguy has quit IRC18:26
heckjthanks18:26
*** heckj has quit IRC18:26
*** heckj has joined #openstack-meeting-alt18:26
kgriffsno problem18:29
kgriffsThought the wiki could say it better than me.18:29
*** yidclare has quit IRC18:36
*** bryansd-home has quit IRC18:38
*** oz_akan has joined #openstack-meeting-alt18:46
*** cp16net is now known as cp16net|away18:46
*** jcru has quit IRC18:46
*** jcru has joined #openstack-meeting-alt18:48
*** yidclare has joined #openstack-meeting-alt18:49
*** edsrzf has joined #openstack-meeting-alt18:51
*** cppcabrera has joined #openstack-meeting-alt18:52
*** bdpayne has quit IRC18:55
*** bdpayne has joined #openstack-meeting-alt18:56
yidclarekgriffs: what's the marconi channel on IRC?18:56
yidclaretried #marconi no-one there.  :)18:57
kgriffsyidclare: #openstack-marconi18:57
yidclarethanks18:57
*** cppcabrera is now known as cppcabrera-away18:58
*** vipul is now known as vipul|away18:58
*** rmohan has quit IRC18:59
*** flaper87 has joined #openstack-meeting-alt19:01
*** rmohan has joined #openstack-meeting-alt19:01
*** cp16net|away is now known as cp16net19:02
*** cp16net is now known as cp16net|away19:02
*** cp16net|away is now known as cp16net19:03
kgriffs#startmeeting marconi19:04
openstackMeeting started Thu Mar 21 19:04:57 2013 UTC.  The chair is kgriffs. Information about MeetBot at http://wiki.debian.org/MeetBot.19:04
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:04
*** openstack changes topic to " (Meeting topic: marconi)"19:05
openstackThe meeting name has been set to 'marconi'19:05
flaper87o/19:05
*** bryansd-home has joined #openstack-meeting-alt19:05
*** bryansd-home has quit IRC19:05
*** bryansd-home has joined #openstack-meeting-alt19:05
kgriffsok, before we get rolling, any items folks want to add to the agenda?19:05
kgriffshttps://wiki.openstack.org/wiki/Meetings/Marconi19:06
*** ametts-atl has joined #openstack-meeting-alt19:06
kgriffs#topic sate of the projects19:07
*** openstack changes topic to "sate of the projects (Meeting topic: marconi)"19:07
kgriffs#topic state of the project19:07
*** openstack changes topic to "state of the project (Meeting topic: marconi)"19:07
kgriffsso, any questions or should I just give a summary?19:08
*** rnirmal has quit IRC19:08
*** malini has joined #openstack-meeting-alt19:08
flaper87kgriffs: summary sounds good19:09
*** rmohan has quit IRC19:09
flaper87some questions might come out of there19:09
kgriffsOK, so we have queue management mostly done19:09
*** rmohan has joined #openstack-meeting-alt19:09
kgriffsThis week we started working on message CRUD19:09
kgriffsWe are trying to get closure on the v1 API, but I think it's stable enough to implement on19:10
kgriffsDoubtless there will be tweaks in the coming months, esp. at the summit19:10
kgriffsWe are close to getting python-marconiclient on stackforge, etc.19:11
*** brc859844 has joined #openstack-meeting-alt19:11
flaper87kgriffs: +2 storage drivers in good progress19:11
*** jrodom has joined #openstack-meeting-alt19:12
*** cppcabrera-away is now known as alejandro19:12
kgriffs(the foundational stuff is in a different repo)19:12
*** alejandro is now known as cppcabrera19:12
kgriffsyes, so we have sqlite and mongo drivers in progress19:12
kgriffssqlite is pretty much for testing, demos, and stuff like that19:12
flaper87kgriffs: perhaps we should send an email to openstack-dev requesting reviews for the python-marconiclient19:12
*** brc859844 has left #openstack-meeting-alt19:13
jdpraxflaper87: Yeah, I just got some additional feedback and a -1. https://review.openstack.org/#/c/24807/19:13
jdpraxThey want me to add pypi configs before merging.19:14
jdprax:-/19:14
jdpraxDoesn't look like a big deal.19:14
flaper87jdprax: cool19:15
kgriffsA few more things I just wanted to mention real quick FYI.  We have 3 core contributors now handling code reviews through Gerrit: Jamie (jdprax), Flavio (flaper87), and myself (Kurt).19:15
kgriffsWe've also been busy adding blueprints, and updating the wiki.19:16
kgriffsAlso, our code is now pep8-friendly, and we have a pretty good HACKING file (still needs a few tweaks)19:16
kgriffsso, there's never been a better time to become a Marconi contributor (hint, hint)19:16
kgriffsOur first big milestone is a demo at the Portland summit19:17
kgriffs(our talk was accepted)19:17
flaper87w000000t19:17
kgriffsso, if you have any cool ideas for a demo give us a shout out in #openstack-marconi19:18
kgriffsOK, any questions before we move on?19:18
* kgriffs listens to crickets chirping19:19
kgriffs#topic API - Maximum queues/page19:19
*** openstack changes topic to "API - Maximum queues/page (Meeting topic: marconi)"19:19
kgriffsOK, so as the blueprints are currently written, Marconi limits queue metadata to 4 KiB.19:20
*** treeder has quit IRC19:20
kgriffsSo, if we allow clients to list 100 queues at a time, that's not a ton of data - should be doable even if you buffer in memory, although we'll likely code such that everything is streamed.19:21
kgriffsQuestions/concerns? https://wiki.openstack.org/wiki/Marconi/specs/api/v1#List_Queues19:21
flaper87sounds good to me!19:22
jdprax+119:22
kgriffsirc://card.freenode.net:7000/#topicĀ API - Maximum queues/page19:23
kgriffssame deal. 4 KiB per message body, 100 messages per request returned19:23
kgriffs#topic API - Maximum queues/page19:23
*** openstack changes topic to "API - Maximum queues/page (Meeting topic: marconi)"19:23
kgriffsOK, so I'll take that as a "+1"19:24
kgriffs:D19:25
flaper87:D19:25
jdpraxAnyone have a good ascii art cricket?19:25
kgriffs#topic API - Claim Messages19:25
*** openstack changes topic to "API - Claim Messages (Meeting topic: marconi)"19:25
kgriffshttps://wiki.openstack.org/wiki/Marconi/specs/api/v1#Claim_Messages19:26
kgriffsso, it would be cool if everyone could read over that real quick and share your thoughts19:26
kgriffsany concerns there?19:29
flaper87kgriffs: just re-read it and seems ok so far19:29
bryansd-homeThe WARNING under Claim Messages, can you explain that a little more?19:29
flaper87I'm still a bit worried about the ttl thing19:29
kgriffssure19:30
bryansd-homeI don't see how you can ever know that the TTL of the claim will be less than that of the messages19:30
kgriffstypically your application would agree on something reasonable.19:30
kgriffsSo producers would always set ttl of messages to X or at minimum X19:30
flaper87kgriffs: so, here's an idea19:31
kgriffsand workers would set claims to some percentage of X, depending on average time to process a given message19:31
kgriffstypically you'd just set the ttl for messages to something like 5 days and the ttl for a claim to 5 minutes19:32
kgriffsflaper87: shoot19:32
flaper87since clients don't know, as bryansd-home also pointed out, what the reasonable ttl would be and we won't be able to "make sure" that messages being claimed wont expire before the claim expires, what if don't make claimed messages expire until the claim expires as well19:34
flaper87so, if the message would expire before the claim expires we would extend the message expiration / ttl 'til the claim expires19:35
jdpraxSounds reasonable to me.19:35
bryansd-home+119:36
kgriffsso, adjust the claim ttl to be min(requested_ttl, max(claimed_messages_ttl))19:36
kgriffssory19:36
kgriffsmax19:36
flaper87kind of. So, if a message expires in 30s and it was claimed and the claim ttl is 60, the message's ttl would be 60s19:37
flaper87we give it a 30s life coupon19:37
flaper87:)19:37
kgriffsoic19:37
kgriffssure19:37
kgriffs+119:38
flaper87cool, place an agreed there19:38
flaper87:P19:38
bryansd-homeI like this. Then a consumer can have some confidence it'll have enough processing time to get through the message.19:38
jdpraxYep.19:38
flaper87bryansd-home: correct, were processing time == claim ttl19:38
bryansd-homeWhereas before it may "claim" a message with only 1s ttl... chaos19:38
flaper87bryansd-home: lol19:38
kgriffs#agreed claim ttls should be adjusted for 30s past the largest age of the messages at claim time19:39
kgriffs#note use age of messages not ttl19:39
flaper87no wait, that's not the idea19:40
flaper87:P19:40
bryansd-homekgriffs: I think we're talking about extending the age of messages, not claims19:40
flaper87bryansd-home: correct19:40
kgriffsoh, oops. got it backwards19:40
kgriffsso, extend age of messages to 30s past expiration of the claim?19:41
kgriffs(if needed)19:41
kgriffsactually, seems like you want more fudge room than that19:42
flaper87if (msg_ttl - msg_age) < claims_ttl: msg_ttl += claim_ttl - msg_ttl19:42
*** vipul|away is now known as vipul19:42
kgriffswell, consider that if workers poll once every, say, 5 minutes for work19:43
kgriffsbut a worker dies.19:43
kgriffsthe claim must expire and the messages still need to live long enough for another worker to pick them up19:43
flaper87kgriffs: nope, because the would expire with the claim (if their life was extended)19:43
kgriffsbut I don't want them to expire with the claim19:44
flaper87kgriffs: they would have expire anyway19:44
flaper87expired*19:44
kgriffsisn't the whole point of having a ttl on claims is so if a worker dies another can pick up the slack?19:44
flaper87kgriffs: sure19:45
flaper87so, the suggestion is:19:46
flaper87Extend messages' expiration time to the claim's expiration time for messages that would expire before the claim does19:47
bryansd-homeflaper87: +119:47
kgriffsno fudge factor?19:47
kgriffsso, clients would still be responsible for doing that themselves if they need it19:47
flaper87kgriffs: clients would set claim's ttl19:48
kgriffsyes, but if a worker client dies and you want a different worker to pick up the unprocessed messages, those messages will need long enough TTLs that they don't expire before another worker picks them up and claims them, thereby extending their lifetimes again.19:49
kgriffsso, setting messages to expire along with a claim only solves the issue of them expiring while a claim is active, not what happens when an agent crashes.19:50
*** amyt has quit IRC19:50
*** amyt has joined #openstack-meeting-alt19:51
flaper87kgriffs: yes but consider that if those message wouldn't have been claimed they would have expired anyway19:51
kgriffsI agree that it is useful to do, and avoids surprises.19:51
kgriffsMy point is just that the latter problem needs to be addressed somehow as well.19:51
kgriffsbasically you need some grace period19:52
kgriffsto ensure messages aren't dropped19:52
kgriffsI wonder if a claim could have a "grace" field?19:52
bryansd-homeOr should message TTLs be renewed when they get claimed?19:53
kgriffsthe the formula would become:19:53
flaper87kgriffs: what I don't understand of your concern is: Why should a claimed message have a grace period and a not claimed message shouldn't ?19:53
kgriffsclaims_ttl: msg_ttl += claim_ttl - msg_ttl + grace19:53
flaper87kgriffs: I'm ok with the "after_claim_grace"19:54
flaper87kgriffs: well, you are right actually. the claimed messages deserve a grace period because the were hold from being processed by the stupid, suicide, worker19:54
bryansd-homehah!19:55
flaper87:)19:55
kgriffsso everyone was right!19:55
flaper87so, +1 for extending and +1 for grace_post_claim19:55
kgriffs#agreed Extend lifetime of claimed messages to be at least as long as the lifetime of the claim itself, plus a specified grace period to deal with crashed workers19:56
kgriffs<phew>19:56
kgriffsOK folks.19:56
flaper87w00000t19:56
kgriffswe have 4 mins19:56
kgriffsopen discussion19:57
flaper87o/19:57
flaper87so, we created now a new milestone series (havana) and a new milestone (havana-1). Marconi is not part of core projects so we're not expected to follow upstream release cycles but it seems to be that it would be good to start "fitting" into those release cycles since now19:58
flaper87so that we're a step ahead when the time will come for marconi to be evaluated19:58
flaper87as a project, organization and blah blah blah19:58
kgriffs+119:59
*** bryansd-home has quit IRC19:59
flaper87there are some upstream milestones we're following (havana-1) and some marconi's milestones (trello) and I think they both can work together19:59
flaper87actually, they would make everything better19:59
*** bryansd-home has joined #openstack-meeting-alt19:59
flaper87so, for anyone with new ideas, proposals or whatsoever, pls, write a blueprint and lets set a milestone to that blueprint based on the workload20:00
kgriffsI guess we could make the summit demo "grizzly-1"20:00
flaper87(which is huge, btw)20:00
kgriffsthen everything after that would be havana milestones20:00
ametts-atlWant to propose Marconi sessions at summit.openstack.org, but there doesn't seem to be a suitable option in the "Topics" drop-down.  (https://wiki.openstack.org/wiki/Marconi)20:01
ametts-atlAnyone know who can help us get an option added to the drop-down?  Or at least a "Incubated Projects" option?20:01
flaper87also, I think it'd be good to get as much feedback as possible from other people20:01
flaper87ametts-atl: mmh, ttx ?20:02
flaper87ametts-atl: Thierry20:02
ametts-atlThierry == ttx?20:02
flaper87ametts-atl: yep20:02
flaper87you'll find him in #openstack-dev20:02
flaper87ametts-atl: at least he can point us to the right direction20:03
ametts-atlYeah, I just posted there.  No response yet.  Maybe I'll try an old fashioned email.20:03
flaper87ametts-atl: sounds better20:03
kgriffsOK folks, anything else?20:03
flaper87ametts-atl: he's based in France, IIRC, so most likely he's doing something else now20:03
flaper87kgriffs: not from me! Great meeting folks20:03
ametts-atlBut you're in Italia, no?!20:04
flaper87ametts-atl: yep, but I'm lifeless and workaholic :P20:04
ametts-atl:)20:04
kgriffsOK, so next community meeting is 4 April20:04
kgriffsin the meantime, folks can catch us in #openstack-marconi20:05
kgriffs#endmeeting20:05
*** openstack changes topic to "OpenStack meetings (alternate) || Development in #openstack-dev || Help in #openstack"20:05
openstackMeeting ended Thu Mar 21 20:05:23 2013 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)20:05
openstackMinutes:        http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-03-21-19.04.html20:05
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-03-21-19.04.txt20:05
openstackLog:            http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-03-21-19.04.log.html20:05
flaper87o/20:05
*** flaper87 has left #openstack-meeting-alt20:05
*** jrodom_ has joined #openstack-meeting-alt20:06
*** bryansd-home has left #openstack-meeting-alt20:06
kgriffscheers20:06
*** cppcabrera has left #openstack-meeting-alt20:08
kgriffsmeeting wiki updated20:08
kgriffshttps://wiki.openstack.org/wiki/Meetings/Marconi20:08
*** kgriffs has left #openstack-meeting-alt20:08
*** vipul is now known as vipul|away20:09
*** malini has left #openstack-meeting-alt20:09
*** jrodom has quit IRC20:10
*** vipul|away is now known as vipul20:11
*** cp16net is now known as cp16net|away20:15
*** cp16net|away is now known as cp16net20:20
*** ametts-atl1 has joined #openstack-meeting-alt20:21
*** ametts-atl has quit IRC20:23
*** cloudchimp has quit IRC20:31
*** demorris has joined #openstack-meeting-alt20:34
*** jrodom_ has quit IRC20:44
*** ametts-atl has joined #openstack-meeting-alt20:58
*** ametts-atl1 has quit IRC21:01
*** jcru has quit IRC21:01
*** esp has joined #openstack-meeting-alt21:01
*** esp has left #openstack-meeting-alt21:01
*** SlickNik has left #openstack-meeting-alt21:02
*** grapex has quit IRC21:14
*** yidclare has quit IRC21:31
*** yidclare has joined #openstack-meeting-alt21:33
*** jdprax has quit IRC21:41
*** edsrzf has left #openstack-meeting-alt21:48
*** oz_akan has quit IRC21:57
*** amyt has quit IRC22:03
*** grapex has joined #openstack-meeting-alt22:09
*** grapex has joined #openstack-meeting-alt22:09
*** grapex has left #openstack-meeting-alt22:12
*** ametts-atl has left #openstack-meeting-alt22:18
*** demorris has quit IRC22:18
*** heckj has quit IRC22:19
*** amyt has joined #openstack-meeting-alt22:22
*** amyt has quit IRC22:30
*** sacharya has quit IRC22:34
*** cloudchimp has joined #openstack-meeting-alt22:36
*** yidclare has quit IRC22:44
*** yidclare has joined #openstack-meeting-alt22:48
*** cloudchimp has quit IRC23:07
*** demorris has joined #openstack-meeting-alt23:18
*** yidclare has quit IRC23:35

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!