14:01:21 <vgridnev_> #startmeeting sahara
14:01:21 <openstack> Meeting started Thu Jun 30 14:01:21 2016 UTC and is due to finish in 60 minutes.  The chair is vgridnev_. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:26 <openstack> The meeting name has been set to 'sahara'
14:01:44 <vgridnev_> hello folks
14:01:47 <egafford> o/
14:01:49 <tosky> o/
14:01:53 <mionkin> hi
14:01:55 <huichun> hello
14:01:57 <johnbelamaric> hi
14:02:01 <crobertsrh> hel\o/
14:02:12 <vgridnev_> i'm experience some issues with my znc, so today i'm vgridnev_
14:02:14 <johnbelamaric> oops - wrong meeting for me - see you :)
14:02:25 <NikitaKonovalov> o/
14:02:29 <zemuvier> hi
14:02:39 <esikachev> hi
14:02:46 <vgridnev_> #link https://wiki.openstack.org/wiki/Meetings/SaharaAgenda
14:02:57 <vgridnev_> #topic News / updates
14:03:37 <egafford> Got the initial version of the image gen CLI up as a WIP; it actually totally works, more or less exactly the way I hoped it would. \o/
14:04:12 <tosky> working on consolidating tests under sahara-tests
14:04:22 <vgridnev_> mostly working on plugins api, and UI for that also
14:04:23 <egafford> I also got an RPM package upstream for sahara-image-elements.
14:04:39 <esikachev> sahara-ci labs reinstalled, updating of sahara-tests patches
14:06:07 <mionkin> added new implementation of migration from novaclient.v2.images to glanceclient; also finished adding CDH 5.7 in sahara and elements (this patches on review)
14:06:44 <zemuvier> working on cli tests
14:07:24 <vgridnev_> huichun, crobertsrh NikitaKonovalov some news?
14:07:47 <crobertsrh> No real news from me.
14:07:58 <crobertsrh> a handful of reviews, but that's it
14:08:20 <NikitaKonovalov> not much from me, I've almost completed the validation patch
14:08:35 <NikitaKonovalov> it is supposed to fix the duplicate names check
14:09:03 <egafford> tellesnobrega: ping?
14:09:17 <huichun> Just reviews ,back to China recently
14:10:27 <vgridnev_> NikitaKonovalov, probably we've got another issue related to is_public field https://bugs.launchpad.net/sahara/+bug/1588795
14:10:27 <openstack> Launchpad bug 1588795 in Sahara "Error when trying to show details of cluster from other tenants (is_public = True)" [High,Confirmed] - Assigned to Vitaly Gridnev (vgridnev)
14:10:55 <vgridnev_> just verified that issue is present on Mitaka lab
14:11:27 <NikitaKonovalov> vgridnev_: I dont think it's related to name validation
14:11:34 <NikitaKonovalov> it looks more like a context issue
14:12:06 <vgridnev_> anyway it's related to public clusters
14:12:24 <vgridnev_> ok, seems like there is no other news
14:13:00 <vgridnev_> #topic Review priorities
14:13:07 <vgridnev_> #link https://etherpad.openstack.org/p/sahara-review-priorities
14:14:44 <vgridnev_> new topic was introduced for core reviewers, so when is change is almost ready to be merged (need one more +2 to be merged) core can put that to bring attention of other cores
14:14:45 <vgridnev_> https://review.openstack.org/#/q/topic:sahara-final-core-vote
14:14:59 <crobertsrh> nice idea
14:15:32 <vgridnev_> additionally, we have changes that should repair your gates
14:15:37 <vgridnev_> #link https://review.openstack.org/#/c/335914/
14:15:56 <vgridnev_> #link https://review.openstack.org/#/c/336024/
14:16:38 <vgridnev_> and the last one for stable branch to address issues with CI
14:16:41 <vgridnev_> #link https://review.openstack.org/#/c/336000/
14:16:54 <vgridnev_> probably it should be also done to stable/liberty
14:17:10 <egafford> vgridnev: On the extjs fix, is this occasionally down or is the file gone forever?
14:17:23 <tosky> given that the last version is 5 and it's commercial, well
14:17:40 <tosky> I suspect that the Hadoop community should try to find a different dependency
14:18:01 <egafford> tosky: +1 certainly, though that's not totally our call. :)
14:18:17 <tosky> yeah, sure :)
14:18:50 <tosky> like support for newer operating systems than "really old centos" and "really old ubuntu"
14:18:55 <tosky> anyway
14:19:07 <egafford> Is there any possibility of hosting the dep from an OpenStack domain?
14:19:21 <egafford> tosky: Yeah, Hadoop is showing its age for sure.
14:20:25 <vgridnev_> egafford, I'll try to ask that with infra folks, probably we can replace other files like Oozie builds, to openstack domain
14:21:10 <egafford> vgridnev_: Yeah, it'd just be a bit more OSS of us. :)
14:22:18 <vgridnev_> so, I think since elmiko is away today, we can skip apiv2 today
14:22:29 <egafford> +1; haven't seen any reviews on it.
14:22:48 <crobertsrh> +1
14:22:48 <vgridnev_> #topic Open discussion
14:23:32 <vgridnev_> no specs to review today, but I hope will get some soon about pagination;
14:23:35 <vgridnev_> not sure
14:23:43 <tosky> I have a work in progress patch that moves the CLI tests under sahara_tempest_plugins
14:24:14 <egafford> Pagination would be good. I also would love to see us get back to talking about rolling upgrades (may try to spec that out after the image CLI.)
14:24:15 <tosky> so I would like to ask if I can proceed with that, I think it's required: https://review.openstack.org/#/c/335946/
14:24:50 <tosky> the downside is that, if accepted, two patches from zemuvier should be wait and be adapted (talking about https://review.openstack.org/#/c/335546/ and https://review.openstack.org/#/c/335959/)
14:25:06 <tosky> note: my patch is not complete yet, there is a failure which I'm investigating
14:25:40 <tosky> if accepted, the plan is to keep the two separate jenkins job for now, but that can be changed
14:25:42 <vgridnev_> egafford, that can be good. we should be back on that, probably blueprint should change assignee of that, since there no activity from ken chen
14:25:56 <egafford> vgridnev_: Yeah, it's pretty critical.
14:26:50 <egafford> tosky: Cool re: your patch (and migrating to tempest for CLI); just ping when it's ready (maybe mark it WIP for now, remove when ready?)
14:27:09 <egafford> (Where by WIP I mean WF-1?)
14:27:26 <vgridnev_> ok, tosky, that might be a good to keep that together
14:27:45 <tosky> also because I'm not sure how to cleanly handle two different tempest entry points
14:27:49 <tosky> and then we can share the configuration
14:28:38 <egafford> Hm; we've lost our chair.
14:28:42 <egafford> \o/
14:28:44 <vgridnev_> oouch
14:28:51 <tosky> and also, packagers (at least for RDO) would scream for another separate directory installed by sahara-tests
14:29:02 <egafford> vgridnev_: (Maybe add a second chair)
14:29:28 <egafford> tosky: Well, we can always subpackage.
14:29:31 <vgridnev_> I just occasionally closed irc client, anyway
14:29:37 <vgridnev_> #chair egafford
14:29:37 <openstack> Current chairs: egafford vgridnev_
14:29:42 <tosky> egafford: trust me, I'd like to avoid a second subpackage
14:31:43 <vgridnev_> there were asked to disable internal storage for sahara https://bugs.launchpad.net/sahara/+bug/1585889
14:31:43 <openstack> Launchpad bug 1585889 in Sahara "There's no option to disable internal db for storing sahara job binaries" [Wishlist,Confirmed] - Assigned to Alexander Ignatov (aignatov)
14:32:24 <vgridnev_> I guess that we can just disable that by option in sahara, anyway, what do with dashboard?
14:33:12 <egafford> vgridnev_: We were talking about removing internal storage entirely now that we have a few better options. Are we still considering that?
14:33:21 <egafford> (Blobs in a DB is really pretty silly.)
14:34:09 <egafford> If we are ok with that, it shouldn't be too crazy to just rip the feature out of the dashboard in all cases.
14:34:45 <vgridnev_> I guess that we can deprecate that in Newton in sahara
14:34:57 <aignatov> egafford: hi, what do you mean by better options? :)
14:35:39 <vgridnev_> aignatov, swift, hdfs, manila, right egafford ?
14:35:49 <egafford> aignatov: Right.
14:36:07 <egafford> (Was writing a longer response, but that's the idea. :) )
14:36:27 <aignatov> vgridnev_: yes, but that are not well tested except swift I guess…
14:36:38 <egafford> I can certainly see leaving the internal DB option for POCs or for the rare customers that really don't want Swift.
14:36:57 <aignatov> egafford: exactly, I think the same
14:37:24 <tosky> yeah, it could be an option disabled by default from Newton and which prints a big warning in the logs when enabled
14:37:53 <aignatov> it’s actully good thing for dev and qa usecases, but not for production for sure :)
14:38:27 <tosky> QEs can afford a simple swift server, or use hdfs :)
14:38:28 <aignatov> I mean lets not rip it, deprecation is ok, to hide it is also ok.
14:38:39 <egafford> aignatov: +1.
14:39:41 <vgridnev_> ok finally, hiding from dashboard by default (by option or always?)  and another option for sahara
14:41:02 <egafford> vgridnev_: I think that's the idea, yeah.
14:41:26 <tosky> do we need an option in the dashboard? Isn't it possible for the dashboard to check if it's exposed in sahara?
14:41:33 <tosky> that would mean maybe an API call
14:42:08 <egafford> tosky: Yeah, building a whole feature architecture would be cool, but I feel like that's very large.
14:42:14 <crobertsrh> Yeah, probably an api call, unless we can piggyback on an existing one
14:42:23 <crobertsrh> I can't think of what we'd piggyback on though
14:42:31 <vgridnev_> uh, I think that is possible to understand from API, if list of internals returned validation error, it maybe a signal
14:42:59 <crobertsrh> heh, I suppose that would work
14:43:02 <egafford> Ideally all of data source, job type, engine, etc. would be plugglable and discoverable, but that's Future Work for sure.
14:44:26 <egafford> vgridnev_: Yeah, that's not actually unreasonable.
14:46:37 <egafford> Anything else today?
14:47:11 <vgridnev_> ok, since we have few minutes, we can also discuss another issue
14:47:14 <vgridnev_> https://bugs.launchpad.net/sahara/+bug/1458853
14:47:14 <openstack> Launchpad bug 1458853 in Sahara "source contains prebuilt java object" [Undecided,Triaged]
14:48:21 <tosky> is there a way to add a build script for those jars? That would allow packagers to rebuild them
14:49:04 <egafford> The sahara.service.edp ones should be okay. Hopefully the MapR folks haven't actually put anything proprietary in their jar.
14:49:14 <vgridnev_> tosky, we have them for sahara-extra, I guess
14:51:00 <vgridnev_> so, we just keeping that on current place, or we can put that on our images? BTW, my only thought is upgrade case
14:51:16 <egafford> Hm; question. If you add the .java files into a .jar with licenses (in addition to the .class files), is it considered OSS enough?
14:51:40 <egafford> That's an easy thing to do that just hasn't been done in these .jars.
14:52:07 <vgridnev_> probably we can put some instructions where to get them and where them are located
14:52:17 <tosky> the point is that there are binary blobs that packagers needs to remove from their packages (and rebuild from source instead)
14:52:49 <tosky> so if there are automated ways to rebuild them that can be used as part of the packaging process, that would be useful
14:53:44 <vgridnev_> so, there is a build script for hadoop-swift, for example
14:53:45 <vgridnev_> https://github.com/openstack/sahara-extra/blob/master/tools/build-hadoop-openstack.sh
14:53:46 <egafford> There are, for sure. Also there aren't any class files in any that aren't org.openstack or org.apache, so we should be fine.
14:54:49 <tosky> I would ask packagers (starting with zigo, who reported the issue) to see what could it be an accepted solution
14:55:59 <vgridnev_> ok, how can start doing that?
14:57:15 <zigo> Use the source luke.
14:57:29 <zigo> Don't build with blobs.
14:59:31 <vgridnev_> it can't be just source, actually.
15:00:12 <egafford> vgridnev_: Maybe we should take this to #openstack-sahara.
15:00:17 <vgridnev_> ok, time is over
15:00:20 <vgridnev_> #endmeeting