17:00:43 #startmeeting app-catalog 17:00:43 Meeting started Thu Dec 17 17:00:43 2015 UTC and is due to finish in 60 minutes. The chair is docaedo. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:45 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:00:47 The meeting name has been set to 'app_catalog' 17:00:55 Todays agenda can be found here: 17:00:57 #link https://wiki.openstack.org/wiki/Meetings/app-catalog#Proposed_Agenda_for_December_17th.2C_2015_.281700_UTC.29 17:01:04 #topic rollcall 17:01:07 o/ 17:01:09 o/ 17:01:10 o/ 17:01:29 hey look at that, it's enough for a meeting :D 17:02:00 #topic Status updates 17:02:29 I've been working on dead link checker (and some related stuff) 17:02:31 I'd like feedback on the "skip assets with active: false" review, I think it's good to go: 17:02:34 #link https://review.openstack.org/#/c/257663/ 17:02:39 This will be used with the dead link checker (to be discussed later in this meeting) 17:02:42 But happy to discuss the method (thanks kfox1111 for pointing out it should be handled in python rather than JS!) 17:02:44 I want to start using this because the dead-link checker is ready (details later), and we could have stale links being automatically discovered in a week. 17:04:00 any other status updates? ativelkov maybe there's something new re: glare? 17:04:28 Yeah, there are some updates on glare 17:04:56 So, the summit's decision on separating the process for Glance and Glare is getting implemented 17:05:13 The spec has been approved and the actual patches are ready and on review right now 17:05:35 This will allow to run glare without a burden of having the rest of Glance deployed, which should be nice for app-catalog 17:05:43 nice! I've been following, would join the weekly meeting but it happens at a time I can't make :( 17:05:54 ativelkov: absolutely, I'm excited about it! 17:06:23 Also, we are working towards API stabilization (addressing API-WG and DefCore concerns ets). The spec is on review 17:06:59 Also, I've got a design for the pluggable DB backends (The feature we discussed with kfox1111 on the summit). I'll submit the spec in a day or two 17:07:28 And last but not least: we are planning a video call in the nearest days to discuss glare roadmap in voice 17:07:46 All are welcome to join 17:08:16 please share the details on the ML and maybe on #openstack-app-catalog because I'd like to join and others might as well 17:08:23 We just need to sync with flaper87 and nikhil to find out the timeslot convienient for all 17:08:26 Yeah, sure. 17:08:48 thx 17:08:49 The problem is that the holiday season is approaching and it may be hard to find a slot equally convinient for all. But we'll do our best 17:09:17 yeah, we are rapidly running out of time to get things done in 2015 :) 17:09:46 some links: 17:09:54 #link https://review.openstack.org/#/c/254710/ - the api refactoring spec 17:10:28 #link https://review.openstack.org/#/q/topic:bp/move-v3-to-glare - patches to move the glare api to separate process / keystone endpoint 17:11:01 Feel free to review and leave comments if you have time :) 17:11:47 ativelkov: thanks, I will 17:12:20 I'll hold for one minute to see if there are any other status updates before moving on... 17:13:38 #topic Dead link check script 17:13:53 This script goes through the assets.yaml file, grabs the URLs and does a header request (with redirect follow). 17:13:56 If it doesn't get a 200 response, it adds "active: false" to the attributes section. 17:13:58 #link http://paste.openstack.org/show/482152 17:14:00 This is the output from the script, which found two dead links! 17:14:03 #link http://paste.openstack.org/show/482151 17:14:14 You'll note one difference from the active assets.yaml we have - the "---" at the top is gone, as that's no longer necessary due to there being only "assets" in the yaml, and no other record types. 17:14:20 It is adapted from the project normalizer script infra uses. It's coupled with a shell script that clones the repo, and if the check_catalog.py script found any dead links, it submits a review to update assets.yaml. 17:14:30 I'll propose this in app-catalog/tools and will coordinate with infra to sort out what user actually submits the PR (probably catalog-bot). With any luck we could have automated dead-url checks live in less than a week! 17:14:37 sweet 17:14:57 I want to use this same script to add a mechanism for keeping a frequently changing asset up to date. The use case would be something like the Debian weekly testing image 17:15:00 #link http://cdimage.debian.org/cdimage/openstack/testing/ 17:15:07 The asset entry would include something like "desired-hash-url: http://cdimage.debian.org/cdimage/openstack/testing/MD5SUMS", and the script could see if the current hash entry in assets.yaml matches. 17:15:09 If it's different, it updates the entry with the hash and an "updated: " entry in attributes section. 17:15:11 The tricky part will be making this abstract enough to work for more than just debian weekly. 17:15:13 What do you think? 17:16:23 makes sense to me... 17:16:47 Well, this allows to ensure that the image is not modified on its way, yes 17:17:19 ativelkov: well yes, but more than that, will allow us to put some frequently changing things in the catalog without having to manually update them all the time 17:17:48 Yes, I understand that. But I believe we should provide some notice to the users 17:17:54 ativelkov: when we have glare for the backend, this will probably change, because it will become reasonable for someone to just use the API to authenticate and update a record they own 17:18:19 Something like "The asset has been changed by its owner since the last time we've checked that" 17:18:57 oh I was actually thinking the entry in the catalog would just get updated to include the new/latest hash when it changes, along with an entry for updated date 17:19:37 that way any time someone comes back to find the latest debian (or coreos, or any of the others that frequently change images) 17:19:51 they see what the latest is, along with a verifiable hash, and the date the record was updated 17:20:08 vs. right now for instance, we just point to the qcow entry and don't list the hash 17:20:09 Ah yeah, so we need two timestamps then 17:20:22 Or may be even three 17:20:48 date when the entry is added to catalog, the date of the last update and the date of the last (manual) verification of the asset 17:21:18 so the clients may choose not to download unverified assets 17:21:46 I would agree with added and updated, but not last verified (because the intention is that this will be automatically verified every day, triggering a review if anything changes - would not be useful to update every record with that date every day) 17:22:13 Keep in mind too - this is a stop-gap thing until we implement glare backend, when many of these mechanisms will change 17:22:27 yeah, you are right, I think 17:22:37 And manual verification does not scale for sure 17:22:49 My intention with this is to take the least-effort path to keeping assets "fresh" until we have a legitimate backend in place :) 17:23:16 I am done with dead-link checker except for coordination with infra, and can probably sort out the "hash checking" stuff before end of year 17:23:44 then next up probably work on some UI refreshes I have in mind while coordinating with you and others on glare implementation 17:24:35 cool 17:24:45 As for manual in my short-term plan, the only manual step will be reviewing what the bot submits when it finds a change. That's low effort for me or others, but also provides a nice safety valve in case the bot goes crazy we can just ignore what it submits 17:25:18 yeah, that sounds the right thing to have 17:25:46 sweet, agreement all around :) 17:26:55 as for the meeting agenda, I left "discuss glare outline" up, but I'm good with the draft and had it there in case kfox1111 and kzaitsev_ws were here (they might have had some points of debate) 17:27:26 So since they're not here, unless olaph or ativelkov want to chat about it? 17:27:28 #link https://etherpad.openstack.org/p/app-cat-glare 17:28:14 I'm good 17:28:31 I did review kzaitsev_ws's etherpad, it was quite aligned with what we are doing in glare 17:28:53 yep, I like the plan there too 17:28:58 once again - agreement all around :) 17:29:06 so moving on... 17:29:14 #topic Open discussion 17:29:48 Anyone have any open discussion points to openly discuss? 17:30:08 * docaedo stops typing for 1-2 minutes 17:30:17 hi 17:30:29 oh hey! 17:31:12 I added a few things to the etherpad, but still need to find a few minutes to finish reading. theres quite a bit in there. 17:32:18 kfox1111: are you implying Community App Catalog is not your only full-time job? What, are you trying to run a huge cloud or something too??? ;) 17:32:39 ;) 17:33:00 Considering the scale of the effort, I'd say we leave it on the agenda and continue to discuss in January 17:33:14 +1 17:33:17 (and of course, on #openstack-app-catalog between now and then) 17:34:10 Agree. 17:35:55 ok - anything else for open discussion? kfox1111, anything you wanted to touch on that we talked about earlier? Much scrollback for you I think... 17:38:49 things look good to me. 17:38:57 sounds like we're done for today then... 17:39:11 yeah, keeping the sums up to date is seperate from prompting users there is updated stuff. 17:39:21 hopefully versioning plays in somehow there too. 17:40:47 in the short term, I don't see any reasonable way to do versions in a manually-updated assets.yaml file, but will HAVE to have versions with glare 17:41:06 I should say .. versions could be done 17:41:15 I'm curious if we can do some kind of web scraper plugin thingy for a few different things. 17:41:21 like, ubuntu's cloud images page. 17:41:45 they update them constantly, and have a lot available. being able to run a cron job that generates bits of assets.yaml with all the versions would be very nice. 17:42:05 I think fedora does something similar. 17:42:28 something to think about. 17:42:40 definitely worth thinking about 17:43:04 (another thing unioning the yaml could help with) 17:43:20 gtg. talk to you later. 17:43:31 ok thanks for dropping by :) 17:43:42 I think we're done - thanks everyone for joining! 17:44:16 #endmeeting