19:01:28 <SotK> #startmeeting storyboard
19:01:29 <openstack> Meeting started Wed Apr 26 19:01:28 2017 UTC and is due to finish in 60 minutes.  The chair is SotK. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:30 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:32 <openstack> The meeting name has been set to 'storyboard'
19:01:58 <SotK> #link https://wiki.openstack.org/wiki/Meetings/StoryBoard#Agenda_for_next_meeting Agenda
19:02:18 <SotK> #topic Announcements
19:02:27 <SotK> Nothing here
19:02:35 <SotK> #topic Urgent Items
19:02:44 <SotK> Also nothing urgent right now
19:02:57 <SotK> #topic In Progress Work
19:03:33 <zara_the_lemur__> I am slowly but steadily reviewing things; somewhat distracted by summit prep
19:04:10 <zara_the_lemur__> some reviews waiting over here:
19:04:11 <fungi> looking forward to the survival guide presentation
19:04:19 <zara_the_lemur__> #link https://review.openstack.org/#/q/project:openstack-infra/storyboard
19:04:23 <zara_the_lemur__> haha, thanks :)
19:04:37 <zara_the_lemur__> #link https://review.openstack.org/#/q/project:openstack-infra/storyboard-webclient
19:04:41 <zara_the_lemur__> I'm still in panic mode
19:05:44 <zara_the_lemur__> fungI: also I've been keeping track of questions that seem to come up a lot and answers to those; I may run the list by you some time at the end of this week in case you can think of any we've missed
19:05:54 <zara_the_lemur__> erm sorry about that strange capital I
19:06:32 <zara_the_lemur__> off the top of my head, 'will bug ids clash', 'what happens to blueprints', 'what's the syntax for gerrit'
19:06:35 <zara_the_lemur__> seem to come up the most
19:06:47 * SotK has also been reviewing things, but sadly failed to test any projects again
19:06:56 <SotK> fungi: thanks :)
19:07:09 <persia> That "where is importance" doesn't come up more often is a testament to the existing documentation.
19:07:32 <zara_the_lemur__> :)
19:07:47 <zara_the_lemur__> I did have a chance to test one import for a bigger project
19:08:10 <zara_the_lemur__> it took a very long time (so long I stopped the import before it finished), so the migration scripts will require optimisations
19:08:13 <zara_the_lemur__> for bigger projects
19:08:22 <fungi> zara_the_lemur__: happy to review your faq skeleton once you have it
19:08:32 <fungi> i agree it's a great idea
19:08:54 <zara_the_lemur__> thanks :)
19:09:58 <SotK> I did some profiling of the scripts a while back because they were painfully slow, pretty much all of the slowness is waiting on the launchpad API
19:10:18 <zara_the_lemur__> yeah, I wonder if we can do fewer requests?
19:10:56 <zara_the_lemur__> if that's not possible, one alternative might be to only import a subset of bugs immediately (eg: only open bugs), and then stagger the others (eg: merged bugs) over a period of days or something
19:11:16 <zara_the_lemur__> it's (very roughly) 1hr per 1000 bugs imported
19:11:45 <SotK> I couldn't see any obvious places to do less requests given how launchpadlib works, but I didn't look entirely thoroughly
19:12:03 <zara_the_lemur__> (though I expect it not to be as straightforward as that, because part of the slowness is around importing users, etc)
19:12:20 <zara_the_lemur__> (and there are likely to be fewer new users to import as the script goes down the list of bugs)
19:13:04 <zara_the_lemur__> (for scale, the project I tested was openstack-manuals; it's about 7000 bugs total. Nova is about double)
19:13:24 <zara_the_lemur__> (that's open, merged and invalid)
19:13:24 <SotK> we'll definitely need to come up with something then
19:13:49 <zara_the_lemur__> yeah
19:14:01 <zara_the_lemur__> I don't think it was a limitation with the hardware
19:14:18 <zara_the_lemur__> (my vm had 4gb ram, on a machine with an ssd and an i7)
19:14:45 <zara_the_lemur__> so there probably aren't easy ways to speed it up around that. :(
19:15:33 <clarkb> could you split the import sets? and have several machines operating on the same db (assuming its an api issue)
19:16:05 <SotK> I expect that would work actually, yeah
19:16:29 <SotK> we'd need to add some way of doing it to the scripts first though
19:16:37 * zara_the_lemur__ wonders how we'd tell the scripts to split it
19:16:40 <zara_the_lemur__> ah snap-ish
19:16:44 <fungi> given that the import is meant to be idempotent, does rerunning to get additional bugs later go more quickly since it can skip the content it already imported?
19:16:55 <zara_the_lemur__> yes, it should be
19:17:36 <fungi> i don't personally care hos long the import takes to complete. if it means we leave the script running for a week to seem most of the bugs and then do some faster passes afterward to get the updates before telling people to switch, that seems good enough to nme
19:17:41 <fungi> er, to me
19:17:49 <fungi> s/hos/how/
19:18:10 <zara_the_lemur__> okay, if you're okay with that, then I'm also fine with leaving it :)
19:18:15 * fungi isn't typing well this afternoon
19:18:16 <SotK> me too
19:18:31 <zara_the_lemur__> so we have 3 known issues with the scripts: handling unusual unicode character codes, erroring out when given any sort of error response code (including a 410 if, say, a user has been banned from lp) and slowness for big projects (which maybe isn't an issue)
19:18:38 <fungi> it's a one-time event, more or less, so not like it needs extreme optimization
19:18:46 <zara_the_lemur__> indeed
19:19:33 <zara_the_lemur__> I thought '16 hours to import a project' might be pushing it, but if it's fine to leave it running then that works for me
19:20:05 <zara_the_lemur__> we had one project import fail because it tried to import an lp user whose name includes a robot emoji
19:20:18 <zara_the_lemur__> and one fail because it tried to import a user who had been banned from lp for spam, raising a 410
19:20:48 <zara_the_lemur__> so we need better handling for that sort of thing
19:20:53 <SotK> the emoji thing should be fairly simple to fix, I'm fairly sure we just need to change the charset the database is using
19:21:45 <zara_the_lemur__> *nod*
19:22:14 <fungi> likely just need to make sure it's using 4-byte instead of mysql's 3-byte default
19:22:37 <SotK> indeed
19:22:44 <fungi> if it only chokes on 4-byte codepoints, that's a sure sign
19:23:10 <fungi> etherpad had a similar issue up until a couple years ago
19:23:54 <fungi> or, rather, our etherpad deployment did (because we were using mysql defaults)
19:24:40 <SotK> yeah, I'd naively assumed that utf8 meant utf8 rather than utf8-that-is-3-bytes-or-less
19:24:47 <zara_the_lemur__> that sounds like fun to discover :)
19:26:04 <zara_the_lemur__> talk planning, review, import script testing... I don't think I have anything else in-progress
19:26:22 * SotK neither
19:26:28 <zara_the_lemur__> I haven't got to kendall's latest branch patch updates
19:26:37 * SotK has, just this evening
19:26:38 <zara_the_lemur__> so will mention those since she's not able to attend today
19:26:41 <zara_the_lemur__> ah, nice
19:27:25 <zara_the_lemur__> ah you should probably -1 so it updates her
19:27:34 <SotK> good point, will do
19:27:46 <SotK> anything else in progress?
19:28:19 <zara_the_lemur__> nope, I'm behind enough on the things I've mentioned already \o/
19:28:24 <SotK> #topic Open Discussion
19:29:38 <zara_the_lemur__> tallllllllllllk
19:29:52 <zara_the_lemur__> as in, boston talk
19:30:28 <zara_the_lemur__> it will be on weds 10th
19:30:29 <fungi> the title is catchy
19:30:53 <zara_the_lemur__> yay, that was diablo_rojo's work :D
19:30:57 <zara_the_lemur__> she'll be pleased
19:31:06 <fungi> i've got it on my shortlist (for sessions i will definitely attend)
19:31:12 <zara_the_lemur__> \o/
19:32:09 <zara_the_lemur__> I'll find the details now for anyone interested, since I'm procrastinating from actually making material for the talk
19:32:42 <SotK> two weeks today!
19:32:49 <zara_the_lemur__> I seem to recall it's weds early afternoon
19:33:15 <fungi> yep, 1:50-2:30pm local time
19:33:24 <fungi> according to my notes anyway
19:33:37 <zara_the_lemur__> #link https://www.openstack.org/summit/boston-2017/summit-schedule/events/17632/storyboard-101-survival-guide-to-the-great-migration
19:34:06 <zara_the_lemur__> we should also be generally around for questions etc over the summit, and available in #storyboard
19:34:10 <zara_the_lemur__> with the usual summit lag ;)
19:34:22 <zara_the_lemur__> (how do people look at irc and talk in person?)
19:34:26 <zara_the_lemur__> (never figured it out)
19:35:08 <fungi> portable computing devices
19:35:17 <fungi> and a lot of inconvenience ;)
19:35:34 <zara_the_lemur__> we live in the future~
19:35:57 <zara_the_lemur__> also, if anyone has been receiving a lot of questions about storyboard and would like them covered, feel free to send them our way
19:36:07 <SotK> +1
19:36:55 <fungi> i'll see if i can round up some i received/answered for you
19:37:02 <zara_the_lemur__> thank you :)
19:37:16 <zara_the_lemur__> I see the 'frequently asked questions' that make it into #storyboard or the mailing list, but I feel like those might not be that representative
19:37:42 <zara_the_lemur__> if people are already informed enough to find their way there
19:37:48 <fungi> diablo_rojo_phon and i have received some questions in private e-mail discussions
19:37:58 <fungi> so happy to help get those merged in
19:38:08 <zara_the_lemur__> cool, I think that would help
19:42:14 * zara_the_lemur__ doesn't think she has anything else to mention
19:42:31 * SotK also has nothing
19:42:39 * diablo_rojo_phon catches up from dentist parking lot
19:42:49 <zara_the_lemur__> that's dedication!
19:42:53 <SotK> hey diablo_rojo_phon :)
19:43:10 <fungi> try not to type with your teeth!
19:43:10 <diablo_rojo_phon> Haha if you say so.
19:43:53 <diablo_rojo_phon> Haha I'll avoid that. Not sure it would work great on the phone anyway.
19:44:23 <diablo_rojo_phon> I saw SotK made comments on my patch that I will address before the end of the week.
19:44:40 <diablo_rojo_phon> I also need to finish writing up the release goal before the end of next week.
19:44:49 <diablo_rojo_phon> And work on slides for our presentation.
19:45:39 <zara_the_lemur__> sounds like a busy week :)
19:45:51 <diablo_rojo_phon> That's only the sb stuff lol
19:46:05 <zara_the_lemur__> heh
19:46:18 <zara_the_lemur__> well we should all be around for the slides
19:46:19 <diablo_rojo_phon> I haven't gotten confirmation from cloudkitty that they want to move, but we can go ahead and do craton if we want.
19:46:31 <diablo_rojo_phon> I just need to give them a heads up about when it will happen.
19:46:51 <SotK> do we want to decide a time/day?
19:47:15 <zara_the_lemur__> guessing we want to rule out summit week
19:47:32 <SotK> I think we do
19:47:36 <diablo_rojo_phon> Add stuff to slides and we should have them done eod Friday so we can go through them next week and break it up.
19:47:49 <fungi> yeah, i'm travelling may 4 through 14 so will likely need to find another infra-root volunteer if they want it on one of those dates
19:48:06 <diablo_rojo_phon> fungi: noted :)
19:48:09 <clarkb> I too am afks from like the 5th to the 16th
19:48:26 <zara_the_lemur__> I expect the end of this week to be a big rush of 'ACK WE ACTUALLY HAVE TO DO A TALK' on our side
19:48:42 <fungi> i already have that x3 or 4 anyway
19:48:49 <zara_the_lemur__> haha
19:49:01 <fungi> need to decide which talks i should panic about first ;)
19:49:11 <zara_the_lemur__> ah, an important decision
19:49:20 <zara_the_lemur__> that will consume all mental energy
19:49:22 <diablo_rojo_phon> Yeah I have sb, upstream training, keynote, foundation fundamentals and ask a stacker shifts...
19:49:23 <zara_the_lemur__> until it's time for the first talk
19:49:27 <zara_the_lemur__> leaving no time to prepare for a talk
19:50:02 <diablo_rojo_phon> fungi: not all simultaneously?
19:50:26 <fungi> diablo_rojo_phon: my multitasking skills are sub-par
19:50:38 <fungi> even panics must be serialized
19:51:05 <SotK> solution: panic over which thing to panic about instead
19:51:24 <zara_the_lemur__> all else fails, improv
19:51:32 <diablo_rojo_phon> SotK: +1
19:52:27 <SotK> anything else to discuss?
19:53:04 <diablo_rojo_phon> Don't think so.
19:53:19 <zara_the_lemur__> at least we're not doing a musical
19:53:22 * zara_the_lemur__ also has nothing
19:53:29 <SotK> ha
19:53:39 <SotK> thanks for coming folks!
19:53:42 <SotK> #endmeeting