15:01:56 #startmeeting StoryBoard 15:01:57 Meeting started Mon Jun 23 15:01:56 2014 UTC and is due to finish in 60 minutes. The chair is krotscheck. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:58 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:02:01 The meeting name has been set to 'storyboard' 15:02:03 Allo allo! 15:02:03 o/ 15:02:10 o/ 15:02:12 hi 15:02:15 o/ 15:02:30 Agenda is here: 15:02:30 https://wiki.openstack.org/wiki/StoryBoard#Agenda 15:02:47 Anyone have last minute additions they’d like to put on there? 15:03:10 * fungi lurks 15:03:17 #topic Reviews: Validate project & group names 15:03:19 I'd like to extend Search UI to Search UI and API 15:03:23 Hi.. 15:03:24 NikitaKonovalov: Got it 15:03:42 I am Aishwarya. 15:03:52 Hi there! 15:03:53 o/ 15:03:59 , a CS grad student from Stony Brook University, the new intern to your team. 15:04:19 To everyone, Ish__ is joining StoryBoard for the summer :) 15:04:25 Ish__: welcome! 15:04:32 woot. welcome Ish__ 15:04:37 Ish__: welcome! 15:04:40 Ish__: o/ 15:04:48 Where is Stony Brook? 15:04:51 Thank you, all..! 15:05:00 * ttx should google that 15:05:01 in NY 15:05:03 NikitaKonovalov: Anything on Project & Group names? 15:05:08 Ish__: cool. 15:05:15 I've added a migration 15:05:28 It will rename a Project if it does not pass validation 15:05:37 NikitaKonovalov: Awesome, thanks for that. 15:05:43 Anything else? 15:05:45 a previous name will be appended to the description 15:05:54 so supersuers can see it and restore 15:05:55 Oh, smart move. 15:05:58 NikitaKonovalov: i like that solution 15:06:07 ++ 15:06:14 NikitaKonovalov: it has the right balance of "keep things working" and "let the admin know something is wrong so they can fix it" 15:06:35 * ttx likes too. 15:07:05 Awesome. Review is here: https://review.openstack.org/#/c/98728/ 15:07:27 #topic Search UI and API 15:08:01 Ok, so I put up a mostly functional multidimensional search UI here: https://review.openstack.org/#/c/99975/ 15:08:21 It still needs some love with no-result edge cases, however the basic feel is there. 15:08:39 Note that right now it only pretends to be a search, it’s really a browse/filter UI. 15:08:54 ok 15:08:58 Seen that, looks great 15:09:25 Also, it’s not the fastest thing in the world, as it waits for multiple search requests to come back before rendering. 15:09:52 I would like some feedback on the UX on that before going much farther, simply as a sanity check. 15:10:04 krotscheck: is taht speed thing a thing that gets helped with search index? or is there a different thing at play? 15:10:28 (I mean, I know you said it's actually just doing browse/filter right now - so that might be part of it?) 15:10:38 mordred: Right now it seems to be mainly network latency that’s the issue. 15:11:17 mordred: See, it’s actually trying to resolve search dimensions via the API, meaning that every time you type a key, it goes out and queries the User/Project/Story API’s all at once. 15:11:19 krotscheck: i think i left comments to this effect in some review (i think it may be the api side), but yeah, having to hit multiple endpoints seems awkward ... 15:11:33 It is. 15:11:54 krotscheck: what kind of flexibility do we have to say violate the rest standards and make a /search?q=... endpoint? 15:11:56 Having said that though, I was able to make the UI work in the way that I wanted it to with the current API features, so that’s good. 15:12:10 jeblair: We have all the flexibility we want? In fact we probably should. 15:12:27 jeblair: That begs the question on browse vs. search though 15:12:39 NikitaKonovalov should probably weigh in on that. 15:13:21 well, we may have browse and search on the same endpoint 15:13:26 Oh? 15:13:36 depending on a q parameter 15:14:08 but I'd preferr to have /search?q= 15:14:18 How would that work if I, say, asked for /stories?q=foo&project_id=22 15:14:33 (since the latter implies a filter and the former implies a query 15:15:32 it sounds like the API kinda wants to ahve both, yeah? like a /search?q= _and_ a /stories?q=foo&project_id=22 for different purposes? 15:15:40 or maybe I'm crazypantshead? 15:15:49 but we want to have q=&k=v to filter 15:16:09 mordred: Well, how do full text searches interact with indexes? 15:16:24 NikitaKonovalov: We don’t necessarily want that. 15:16:35 http://sqlalchemy-searchable.readthedocs.org/en/latest/ 15:16:55 ^^ can wratp filter queries with add full-text stuff 15:16:58 krotscheck: oh, we'd need the indexingy things to implement /search?q= in a truly performant way - but we could have a v1 of what the API wants to look like without the goodness probably 15:16:58 …postgres only? Srsly? 15:17:15 krotscheck: the python community has an unnatural obession with postgres 15:17:32 mordred: The python community has an unnatural obsession with python :-P 15:17:38 krotscheck: :) 15:17:41 rewrite it all in go! 15:17:47 in javascript!: 15:17:56 Postgres is for an example there 15:17:58 I recommend we rewrite it all in chicken 15:17:59 i hope 15:18:38 NikitaKonovalov: it seems worthwhile to explore whether or not that will function with mysql 15:18:47 mordred: agree 15:18:59 oh, i also left some nodes in a review about the full-text indexing... i think we may want to consider using the messaging framework to keep solr up to date near-real-time 15:19:08 oh. I guess "Currently it only supports PostgreSQL." is what krotscheck was reading 15:19:13 * krotscheck is honestly not that familiar with how search indexes work, as long as the API behaves sanely he’ll happily stay on the client side and let everyone else argue it out. 15:19:44 krotscheck: yah - I mean, honestly, I think we should get a search api written even if the impl is terrible so that it doesn't have to be faked ui side 15:19:44 I've updated the Search spec with Spihnx section 15:19:59 and then we can try different api server impl choices. 15:20:00 it can keep it's indexes pretty up to date 15:20:17 * mordred also agrees with jeblair on messaging framework for keeping search indexes up to date 15:20:26 jeblair: You mean the messaging async processing spec I put up there? 15:20:33 krotscheck: yep 15:20:34 this is what their indexes up to date http://sphinxsearch.com/docs/current.html#delta-updates 15:20:59 looks like we will need messaging for both indexing and notifications 15:21:13 krotscheck: since the search spec seemed to be reaching for a way to get async notifications that it needs to reindex stuff 15:21:19 Before we go off the rails here, can we focus on what decisions actually need to be made right now? 15:21:51 I think I agree with mordred that “implementing the search API no matter what the impl behind the scenes is” is good. 15:21:57 ++ 15:22:20 However, I also feel that the UI I put together buys us more time on the need for this implementation, because it actually provides a pretty sophisticated way of discovering things. 15:22:25 ++ 15:22:32 then the LIKE statement is the solution 15:22:39 Right 15:23:21 It doesn’t remove the need for a fulltext search, it only patches the pain 15:23:38 So it sounds like we want more comments and discussion on Nikita’s review 15:23:57 The real decision to make is: Separate search API endpoint, or munge it in with browse? 15:24:35 pecan allows to do that 15:24:53 I've done a similar thing for GET projects 15:24:57 krotscheck: what's the benefit of munging it with browse? 15:25:34 mordred: Can’t think of one. I feel that it’s mostly a stylistic choice. 15:25:44 Well 15:25:47 Philosophical 15:25:56 krotscheck: fundamentally, i think we want one endpoint to handle the one operation -- what i'm not sure about is whether it needs to be completely separate, or if it's enough to hang it off of stories because that's ultimately what we're going to display to the user... 15:26:15 jeblair: point 15:26:45 jeblair is suggesting search as a resource-wide action, not just specific to individual resources. 15:26:49 Sorry 15:26:52 API-wide action 15:27:47 mordred: Philosophical difference is as follows- search is inclusive: “Give me things that are close to this value”. Browse is exclusive: “Only give me things where this == value“ 15:28:23 NikitaKonovalov: Thoughts about /search?q= vs /stories/search?q= 15:28:40 my vote is for the second 15:28:58 that one clearly says what are we looking for 15:29:22 Doesn’t searching imply that we don’t actually know what we’re looking for? 15:30:11 so then we need an ability to limit the search if the user actually knows what he is looking for 15:30:36 search?q=&target=stories 15:30:37 NikitaKonovalov: Right - at that point it becomes a munged search/browse 15:31:08 krotscheck: maybe we need to lay out some use cases and say "if you want to find a specific comment, do this" "if you want to find a story that has a comment with text, do this".... 15:31:22 jeblair: That’s a good idea. 15:31:25 ++ 15:31:37 krotscheck: i think doing straw-man versions of cases (especially edge cases) like that may be instructive 15:32:06 jeblair: You know, i can go back into the Atlanta UX sessions and extract some of the search cases that were highlighted there. 15:32:19 krotscheck: sounds like a good source 15:32:22 Indeed 15:33:05 Do we need to pause the spec while we figure that out? 15:33:16 Or can the spec be nonspecific as to where the endpoint lives? 15:33:44 i think this would be good stuff to get into the spec 15:33:50 Alright, so high priority. 15:34:06 NikitaKonovalov: Do you mind if I add the use cases to your spec directly? 15:34:18 krotscheck: I'd appreciate that 15:34:25 Works for me. 15:34:41 #action krotscheck Extract search use cases from UX sessions and add to search spec. 15:35:09 SPeaking of specs.... 15:35:13 #topic Specifications 15:35:29 We just talked about Search: https://review.openstack.org/#/c/97254/ 15:35:48 NikitaKonovalov added a section on solr. 15:35:55 We’ve got some use cases to add 15:36:08 And an outstanding decision on where the endpoint should live. 15:36:19 endpoint(s) 15:36:24 Did I miss anything? 15:36:30 I'll remove that sqlalchemy-searchable section 15:36:38 we do not have postgress 15:36:47 Right 15:37:04 Anything else? Else I’ll move on. 15:37:14 NikitaKonovalov: btw, in the commit msg, you can have "Story:" and "Change-Id" in the same paragraph (like in https://review.openstack.org/#/c/95307/ ); they don't need a blank line between them 15:37:41 jeblair: not a problem 15:37:48 jeblair: I dunno, I feel that two lines is more legible for humanz 15:38:04 My meat brain is bad at regex :) 15:38:08 krotscheck: definitely 2 lines, just saying not 3 lines 15:38:12 Oh, got ti 15:38:21 * mordred votes for 5 lines 15:38:35 There’s also the tagging spec: https://review.openstack.org/#/c/97211/ 15:38:50 Which I know I promised I was going to write a sibling for, but I haven’t gotten around to it because of the search UI 15:38:57 krotscheck: we should also make sure to get sdague and clarkb to look over the search spec and provide feedback on the fulltext engines 15:39:30 jeblair: Last time I asked clarkb on that his response was “Oh god not elasticsearch please god not that" 15:39:37 * krotscheck may be paraphrasing 15:39:39 yeah, that's good feedback to have :) 15:40:00 that sounds like something he'd say 15:40:24 #action NikitaKonovalov Get sdague and clarkb’s feedback on Search Spec. 15:40:39 Looks like we have a new update on tagging 15:40:48 ttx: Care to summarize? 15:42:07 Mostly clarifications 15:42:17 Alright, we can read those on our own time. 15:42:31 The messaging spec is here: https://review.openstack.org/#/c/95307/ 15:42:32 on what's accepted in a tag name, and how protected tags would work 15:42:54 jeblair: I left a comment for you regarding the actual implementation that would make a streaming API easy 15:43:06 krotscheck: it still targets a story-specific case, so if you want to introduce a more general reousrce tag, you can file a separate one 15:43:12 krotscheck: cool, thx 15:43:18 ttx I will. 15:43:40 jeblair: That comment will also address the question of updating any indexes. 15:44:24 (Well, it addresses the question of “Hey we have a stream of changes, what useful things can we do with that?”) 15:44:35 #topic Ongoing work 15:44:47 In alphabetical order this week! 15:45:02 I was mostly working on the search UI, as well as tag completion. 15:45:24 This week I’m going to write all the specs and try to advance the storyboard puppet module changes 15:45:40 So that we can get rabbit onto the SB server and we can start broadcasting changes. 15:46:03 NikitaKonovalov? 15:46:42 I've been working on 4xx codes support for UI and API 15:47:06 now 401 and 403 should work properly 15:47:17 Awesome 15:47:19 woot 15:47:24 * krotscheck is surprised that’s not baked into pecan 15:47:37 wasn't there discussion at some point of pecan growing that? 15:47:42 dhellmann: ^^ ? 15:48:06 krotscheck: I'd say I had to fight agains pecan mechanisms 15:48:13 dhellmann: Context: https://review.openstack.org/#/c/101146/ 15:48:29 on rendering the correct code 15:48:34 NikitaKonovalov: Yeah, I looked at that review and was somewhat surprised that there wasn’t a more elegant way 15:48:48 but still that works 15:49:25 and there is a new modal for UI to warn a user if cant perform some action 15:49:35 https://review.openstack.org/#/c/101867/ 15:49:41 Oh neat! 15:50:21 I've done some one-line fixed for timeline and docs 15:50:23 NICE. Love the implementation 15:51:12 and there is a WIP change for search endpoints, but it'll wait until we decide how the final interface should look like 15:51:21 ttx? Thanks for the updated spec and for your reviews, anything in addition to that? 15:51:28 no 15:51:53 Ok, moving to open discussion 15:51:55 slowly working to the bottom of my TODO pile though so I still hope to be able to propose some code to keep current 15:52:01 I stand corrected. 15:52:30 ttx: I could use some help with oslo.messaging :) 15:52:50 Anything else? 15:53:00 nope 15:53:03 #topic Open Discussion 15:53:39 There is going to be an Infra-qa meetup soon 15:53:47 any StoryBoard stuff there? 15:54:14 You mean the midcycle in Dortmund? I won’t be able to make it. 15:54:26 darmstadt 15:54:33 neither will I 15:54:38 Well, i had one letter right! 15:54:53 so we probably won't do much storyboard stuff then, except perhaps to talk about bulk import 15:55:06 since that's the main blocker of increased infra adoption right now 15:55:08 bulk import ftw 15:55:37 Indeed. That discussion was last left with “maybe doing a direct-to-db dump would be best to ensure associations are maintained" 15:57:00 good to know 15:57:12 That’s from the summit 15:57:30 we may end up pinging people in irc if we get to talking/hacking about it 15:57:55 jeblair: Works for me. 15:58:02 Anyone have anything else they want to bring up? 15:58:12 nothing from my side 15:59:59 Ok, thanks everyone! 16:00:01 #endmeeting