15:01:37 #startmeeting Zaqar 15:01:39 Meeting started Mon Oct 6 15:01:37 2014 UTC and is due to finish in 60 minutes. The chair is flaper87. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:41 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:43 The meeting name has been set to 'zaqar' 15:01:55 #topic roll call 15:01:58 who's here? 15:02:02 I'm not 15:02:50 o/ 15:02:52 kgriffs: cpallares vkmc 15:03:01 so, vkmc is traveling, I guess she's not joining 15:03:23 zhiyan: ? 15:03:32 o/ 15:03:42 aaaaaaaaaaaaaanyway, lets move on 15:03:50 This is our agenda for today: https://wiki.openstack.org/wiki/Meetings/Zaqar#Agenda 15:04:09 #topic Actions from previous meetings 15:04:16 so, there are no actions from previous meetings 15:04:24 LOL, that was short, easy and painless 15:04:30 #topic What is our release plan for v1.1? Is it frozen? (kgriffs) 15:04:33 kgriffs: floor is yours 15:05:18 kgriffs: I promise, you'll have breakfast after this 15:05:25 heh 15:05:26 kgriffs: speak, don't be afraid 15:05:28 :P 15:05:52 so, I just wanted to talk about our plans around finalizing v1.1 of the api 15:06:42 afaict, it's pretty much complete now and what we wanted to do on it, it's done 15:06:53 I think we can go ahead and freeze it 15:07:08 are all the pooling things stable now? 15:07:33 yeah, there are some patches with fixes but I think it's fine to accept bug fixes 15:07:45 no new features or changes to the public semantics should be needed 15:08:12 kgriffs: the 2 patches I mentioned add `links` to the returned document for pool and flavor 15:08:21 there's nothing else as far as I remember 15:08:50 what is the strategy for dealing with "additive" changes, such as returning a couple more values in the queue stats response 15:09:26 i can't remember if we have a guideline in the API that says clients must ignore extra fields they don't recognize - in other words, don't break if the response has more things in it 15:09:32 as long as they don't change the existing semantics (meaning they don't break backwards compatibility) I think it's fine 15:09:43 exactly 15:10:24 "Unrecognized protocol elements received from the server should simply be ignored. This includes JSON object properties, link relation types, media types, etc." 15:10:34 #link https://wiki.openstack.org/wiki/Zaqar/specs/api/v1.1#Resource_Media_Types 15:10:58 +2 15:11:59 flaper87: is there anything left we need to get into the final Icehouse release so that if someone deploys that, they can be sure that it will support the entire v1.1 API as spec'd? 15:12:34 mmh, Icehouse? 15:12:41 you meant Juno ? 15:12:44 oops, yeah 15:12:45 * flaper87 confused 15:12:47 ah ok 15:12:48 juno 15:13:05 then no, I think the whole spec is covered 15:13:11 we can do a sanity check this week 15:13:22 and I'll write down a notice announcing v1.1 is frozen 15:13:48 I think even the client has full support for v1.1 now 15:13:50 ok. related, perhaps we can get this on today's agenda too - smoke testing both v1.1 and v1.0 with "recommended" production deployment configurations 15:14:17 that should uncover any minor things that we may have missed 15:14:21 #action flaper87 and kgriffs to sanity check that v1.1 is fully implemented as written in the spec 15:14:43 #action flaper87 to write a note announcing v1.1 is frozen if the previous action is indeed complete 15:14:51 kgriffs: want to do it right away ? 15:14:58 #topic smoke test v1.1 and v1.0 15:15:05 Hi, sorry I'm still in holiday, but for 1.1 Api stuff May I do a double check here? 15:15:17 zhiyan: that'd be great 15:15:25 lets talk when you're back 15:15:30 I'll point you to the right places 15:15:37 Ok 15:16:16 Actually just want to make sure the extra queue stat what I proposed in last meeting 15:17:05 If it could be added in 11 or four should do it in 20 15:17:06 zhiyan: sure, we can talk about that further in a bit if you're still around 15:17:22 I think v1.1 is fine, it's an addition to the stats dict 15:17:27 kgriffs: around? smoke test? 15:17:34 yep 15:17:57 so, I'd like us all to be confident that someone deploying juno in production won't find any bad bugs 15:18:13 Ok. I talked that with kgriffs and flwang, they think these two items are ok 15:18:18 therefore, I propose standing up some clusters and banging on them manually - using curl, the client, whatever 15:18:35 kgriffs: would something like this help? https://review.openstack.org/#/c/125562/4 15:18:44 kgriffs: we can make that job run for zaqar too 15:18:54 I'm adding it to zaqarclient first 15:18:59 so we can test it 15:19:18 but we could make it vote in zaqar and that'd be like our tempest++ test 15:19:33 it'd ensure we're not breaking the client and that we're not breaking the server 15:19:43 that patch runs all the client functional tests 15:19:43 i think that definitely helps. but before each big release I think we need to do some manual testing and against clusters that are outside devstack. 15:19:56 kgriffs: fair enough 15:20:29 kgriffs: any idea how we can automate this? I know you said manual but we need some scripts that would help on making this process easier 15:20:52 mmm, good point 15:22:19 i was just going to say 15:22:36 you can't automatically verify that error messages are helpful/make sense 15:22:47 but then to test that we would need to force error conditions 15:23:00 kgriffs: mmh, ok, then lets organize a test-day 15:23:04 actually, we should do that anyway - like force a master failover and stuff 15:23:07 but anyway 15:23:35 Lets write down a list of tests we want to do 15:23:40 it may not be as much "manual" as "real production deployment and simulating failures" 15:23:43 then organize a test day and call for volunteers 15:23:54 and ask each of them to take just 1 task 15:23:58 and also "requests from a type of client we don't normally test" 15:24:08 we can share this info in openstack-dev and even #rdo 15:24:22 test day sounds good 15:24:28 But I think we first need to work on the list of things we want to test 15:24:35 the biggest amount of work is setting up the test cluster(s) 15:24:39 Make sure it's broad enough to cover a real production environment 15:24:47 flaper87: yep 15:24:56 kgriffs: I can help you with that 15:25:04 also, I think we need the test cluster for some tests 15:25:17 but other tests can also be run manually in each volunteer's laptop 15:25:28 Like verify the usefulness of error messages 15:25:37 well, actually, the test cluster makes everything easier 15:26:01 #action kgriffs and flaper87 to work on a list of pre-release tests for test day 15:26:11 sounds like a plan 15:26:23 kgriffs: awesome, thanks for bringing this up 15:26:38 sure thing 15:26:39 #topic Where are we from a durable/reliable perspective? 15:26:57 So, I'm not going to get much into detail here but I did want to bring this up 15:27:26 I think we need to do a deep dive and analize where we're standing w.r.t durability, reliability and scalability 15:27:40 I'm not talking about performance testing but deployment architectures 15:27:53 for example: https://review.openstack.org/#/c/125938/ 15:28:00 I started working on that documentation 15:28:16 It helped me to clear out where the mongodb driver is standing w.r.t the above 15:28:30 and it'll also help deployers to understand what each driver is useful for 15:28:37 I think we also need to do that for the redis driver 15:28:57 the actual documentation is here: https://review.openstack.org/#/c/125938/4/zaqar/queues/storage/mongodb/__init__.py,cm 15:29:33 Basically, what I'd like us to do is to work on defining what the recommended deployment scenario is from the bottom up 15:30:14 I'm sure more experienced dev/ops will show up and prove us wrong but until then lets provide at least a starting point 15:30:30 should we come up with a set of things that should be documented for each driver? 15:30:32 flaper87: ++ 15:30:58 now, easy question to make but hard to answer: Who volunteers for the redis driver? 15:30:58 like, "here is how you get HA" 15:31:03 kgriffs: I started working on this: https://etherpad.openstack.org/p/zaqar-storage-requirements 15:31:09 and "here is how you scale up and out" 15:31:26 that's kinda something drivers must provide but also a starting point for these docs 15:31:32 and "performance tips" 15:31:50 kgriffs: +1 for performance tips 15:31:54 all those things are useful 15:32:19 I think the main 3 things that must be documented are in that etherpad. We can have custom sections per driver if needed 15:32:52 ok 15:33:04 obviously, feel free to edit that etherpad 15:33:22 it's a work in progress for this patch: https://review.openstack.org/#/c/125658/ 15:33:31 ok, that's it from me for this topic 15:33:37 ok 15:33:43 real quick 15:33:46 shoot 15:34:07 I know everyone is all keen on "scale-out" but there are some "scale up" things that make sense too 15:34:14 for example, use SSDs in your mongo box 15:34:56 indeed there are, I think we have to mention those 15:35:04 mongo docs mention some scale-up tips 15:35:05 I was just thinking it would be good to just mention a few things like that, or at least link to some docs or books people can reference to learn more 15:35:11 and it's fair to point people there 15:35:17 kgriffs: ++ 15:35:39 kewl 15:35:52 I'm also thinking about adding a note in mongo docs saying: I've no idea how sharding works so it'll probably blow up 15:36:01 * flaper87 is obviously kidding 15:36:04 ... about the note 15:36:05 lol 15:36:06 :P 15:36:21 aaanyway 15:36:23 moving on 15:36:28 #topic Design Sessions (flaper87) 15:36:47 sooooooooooo, design sessions https://etherpad.openstack.org/p/kilo-zaqar-summit-topics 15:36:50 #link https://etherpad.openstack.org/p/kilo-zaqar-summit-topics 15:37:18 I'd like to organize more than 4 sessions. We'll have room for 4 and then just pod sessions 15:37:28 but I'd like to give our pod discussions some schedule 15:37:35 just to make sure we know when to be there 15:37:54 that said, I'd like to reserve the design sessions for topics other projects may be interested on 15:38:20 so far I have "v2 kick-off" and "persistent transports" 15:38:32 the summit is less than a month away and we need to sort this out 15:38:49 please, dedicate some time to this and add ideas there 15:38:53 that's it 15:39:06 anything else? anyone? 15:39:42 ok 15:39:42 um 15:39:46 wait 15:39:49 * flaper87 waits 15:40:04 when are Zaqar's 4 slots scheduled? 15:40:12 Tuesday 15:40:18 I have to admit, I've been a bit out of the loop lately 15:40:22 all incuabted projects are scheduled on Tuesday 15:40:35 along with workshops and cross-project things 15:40:41 oic 15:40:47 tuesday morning or afternoon? 15:40:53 is this posted somewhere? 15:40:55 #link http://kilodesignsummit.sched.org/ 15:40:59 both 15:41:08 2 in the morning and 2 in the afternoon 15:41:09 oic 15:41:18 for some reason I thought we weren't using sched this year 15:41:56 oic, we are using it, just the topics are TBD 15:42:14 yup 15:43:18 kgriffs: want to add something else? 15:43:20 :D 15:43:39 I'm just trying to think of what other things would be good to use the room for vs. the pod 15:43:55 do we want to meet up with horizon and heat? 15:43:57 we can do it this week and decide next week 15:44:10 ...and swift 15:44:12 probably heat, I gotta talk with them and see if they want to do that 15:44:44 ok 15:45:07 moving on 15:45:13 #topic Make FIFO optional (flaper87) 15:45:30 first, we obviously don't have to decide now 15:45:31 :D 15:45:34 #link https://review.openstack.org/#/c/125986/ 15:45:45 I submitted that, lets make sure we give it enough discussion 15:45:54 lets comment on the spec and decide there 15:46:14 but lets do it before the summit since that'll be part of the v2 discussion and probably other discussions 15:47:01 OMFG, I just dicovered something 15:47:06 #link http://docs-draft.openstack.org/86/125986/1/check/gate-zaqar-specs-docs/5292dc3/doc/build/html/ 15:47:29 if the docs gate passes you can go to the rendered docs by clicking on the job link 15:47:38 I had no idea 15:47:59 #link http://docs-draft.openstack.org/86/125986/1/check/gate-zaqar-specs-docs/5292dc3/doc/build/html/specs/kilo/fifo-optional.html 15:48:00 pretty 15:48:15 damn, you beat me 15:48:36 ok, lets think about it and also consider what kgriffs wrote here https://etherpad.openstack.org/p/zaqar-design-constraints 15:48:40 ok, so comment on the RST file directly for the discussion 15:48:50 everything must be taken into account 15:48:51 kgriffs: yup 15:49:10 kewl, we've 10 mins left 15:49:16 anything else on this topic? 15:49:26 I'm good 15:49:35 #topic open discussion 15:50:08 kgriffs: I saw you commented on the sentinel patch 15:50:21 lets see if it gets udpated soon enough 15:50:26 otherwise, we should do it 15:50:35 I'd like to talk with ttx about the rc2 asap 15:51:19 yeah, looks like no new patchsets since we talked about it last week 15:51:42 kgriffs: ok, I'd say feel free to update it 15:52:32 ok, if someone sees prashanth please let him know 15:52:55 I will look for him tomorrow morning! 15:53:00 ok, that's all folks 15:53:07 flaper87: 15:53:08 wait 15:53:12 * flaper87 waits 15:53:13 what else should go in RC2 15:53:13 :P 15:53:25 are you keeping a list of things to backport? 15:53:30 #link https://review.openstack.org/#/c/123750/ 15:53:31 yes 15:53:34 just 2 patches 15:53:39 sentinel's and that one 15:53:58 I'll move bugs to rc2 as soon as I have a milestone for it 15:54:03 what about 15:54:04 I gotta talk with ttx 15:54:06 #link https://review.openstack.org/#/c/123626/ 15:54:32 and 15:54:35 #link https://review.openstack.org/#/c/123462/ 15:55:02 yeah, those 2 are rc2 worth it. I wasn't *that* worried because they are not release blockers and can be backported even after the release 15:55:11 wherease the sentinel thing is indeed release blocker 15:55:20 but since we'll have rc2, we can backport all these patches 15:55:35 that should give us a cleaner release 15:55:41 ok 15:55:46 that's it from me 15:56:44 sweet :D 15:56:51 phew, we covered quite some things 15:57:05 ok, that's all folks, for realz this time! 15:57:10 tty all next week 15:57:13 have a great week 15:57:16 #endmeeting