18:05:19 <SergeyLukjanov> #startmeeting sahara
18:05:20 <openstack> Meeting started Thu Apr 10 18:05:19 2014 UTC and is due to finish in 60 minutes.  The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:05:21 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:05:24 <openstack> The meeting name has been set to 'sahara'
18:05:27 <SergeyLukjanov> #ling https://wiki.openstack.org/wiki/Meetings/SaharaAgenda
18:05:34 <SergeyLukjanov> hey folks
18:05:49 <SergeyLukjanov> traditionally, the first topic is....
18:05:51 <SergeyLukjanov> #topic News / updates
18:05:55 <SergeyLukjanov> folks, please
18:06:25 <dmitryme> I was mostly finalizing the docs
18:06:30 <crobertsrh> I've started the merging process for the dashboard into horizon.  I hope to have all the CRs up no later than tomorrow.
18:06:40 <mattf> sahara is going to get air time during #devnation next week (assuming i start & finish the presentation). also, it'll get air time during hadoop summit in san jose.
18:07:07 <ylobankov_> I am testing all things related with Sahara
18:07:12 <aignatov> mattf: cool!
18:07:53 <mattf> aignatov, i may snag your todo demo for #devnation. i can't seem to get my sigmask in place to make something.
18:08:25 <aignatov> I was working a few on docs and tested all related heat stuff in sahara, also updated sahara's resource in heat
18:08:30 <mattf> re conference - has anyone submitted to hadoop world in nyc?
18:08:53 <aignatov> mattf: you mean demo about "Top todoers"?
18:09:18 <SergeyLukjanov> crobertsrh, awesome, could you please announce new CRs to #openstack-sahara?
18:09:37 <aignatov> feel free to show it for masses :)
18:09:40 <mattf> aignatov, quite possibly
18:09:45 <crobertsrh> #link  https://review.openstack.org/#/c/86648/
18:09:51 <SergeyLukjanov> #link https://etherpad.openstack.org/p/sahara-icehouse-tbd
18:09:58 <crobertsrh> #link https://review.openstack.org/#/c/86614/
18:10:10 <mattf> aignatov, i'm thinking about adding a time component. who's removing their todos vs who is just accumulating them
18:10:40 <SergeyLukjanov> #info the blocker for RC2 is HDP features / desciption update https://review.openstack.org/#/c/84797/
18:10:49 <SergeyLukjanov> when it'll be merged, we're reading to rc2
18:10:58 <dmitryme> mattf: sounds like an interesting variation
18:11:05 <SergeyLukjanov> I'll double check that all needed CRs are backported to m-p
18:11:11 <mattf> dmitryme, i'll look pretty bad in it
18:11:18 <SergeyLukjanov> and ask ttx to push the rc2 tag
18:11:24 <mattf> dmitryme, could be good for public shaming
18:11:36 <aignatov> mattf: nice idea, I mean to get sources for one date and sources for another date and then a diff in TODOs, right?
18:11:59 <dmitryme> SergeyLukjanov: can we put https://review.openstack.org/#/c/86641/ into rc2 as well?
18:12:01 <mattf> aignatov, for speed i might just pull release tarballs
18:12:05 <tmckay> all, I'm pursuing the bigpetstore app as a demo, but it's not there yet.  2 parts, data generation, and csv processing.  I ran the first part (with hdfs, needs mods to use swift).  The 2nd part launches PigServer from inside a java app, so I'm trying to run it as a Java action but it needs a pig lib on the classpath.  The other option is to extract the Pig script built in the java app and allow Oozie's pig runner to work on that.
18:12:22 <aignatov> probably this idea is a good on more demo's candidate for Atalanta summit
18:13:10 <mattf> ideally we can have a different demo for #devnation, os summit, hadoop summit
18:13:11 <SergeyLukjanov> dmitryme, sure, nice tip
18:13:39 <SergeyLukjanov> mattf, aignatov, jspeidel, alazarev, please, review/approve https://review.openstack.org/#/c/86641/
18:13:56 <mattf> SergeyLukjanov, i've been super distracted, but at some point i'd like to formalize the criteria for breaching feature freeze or rc
18:14:38 <SergeyLukjanov> mattf, it was already defined - hadoop2 updates and docs
18:14:59 <sreshetnyak> o/
18:15:25 <mattf> SergeyLukjanov, my sampling of merges suggests we're not sticking to that. your notion of the criteria must not match my notion.
18:15:27 <aignatov> yep, seems to be hadoop 2 related changes are already in the icehouse
18:15:34 <aignatov> only docs left
18:15:46 <SergeyLukjanov> mattf, and open issue, that are blocking some important features
18:15:50 <SergeyLukjanov> #info https://launchpad.net/sahara/+milestone/icehouse-rc2
18:16:16 <mattf> important is subjective
18:16:39 <SergeyLukjanov> mattf, note that master is Juno already, not everything merged to master backported to m-p
18:17:14 <mattf> SergeyLukjanov, /me nods
18:17:50 <mattf> i'd like to discuss at some point, but after next week...
18:18:18 <aignatov> release will be on the next week :)
18:18:26 * mattf nods
18:18:32 <SergeyLukjanov> mattf, we'll have much more for this process next release
18:18:43 * mattf breaks neck nodding
18:19:19 <SergeyLukjanov> mattf, this time we have last two weeks before FF renaming savanna to sahara instead of doing things that we're doing now
18:19:49 <SergeyLukjanov> so, that's why I've approved more backports than I wnat to see during the rc perion
18:19:50 <mattf> SergeyLukjanov, i'm just raising a warning flag that we should find a way to formalize the criteria
18:19:53 <SergeyLukjanov> period*
18:19:58 <mattf> and clearly it's too late for this release
18:20:18 <SergeyLukjanov> mattf, there is no way to formalize it ;) take a look on other projects
18:20:35 <mattf> well, at least socialize it then
18:20:50 <mattf> but...we should move on
18:21:13 <SergeyLukjanov> all of the stuff was added to etherpad and milestone that were announced and discussed several times in irc and ml
18:21:41 <SergeyLukjanov> oh, I find useful link
18:21:44 <SergeyLukjanov> #link https://github.com/openstack/sahara/compare/2014.1.rc1...milestone-proposed
18:21:50 <mattf> i really don't want to entirely derail the meeting w/ details
18:22:57 <SergeyLukjanov> so, for rc2, we're waiting for https://review.openstack.org/#/c/86641/ https://review.openstack.org/#/c/84797/ and https://review.openstack.org/#/c/83036/
18:23:06 <SergeyLukjanov> any objections?
18:23:11 <SergeyLukjanov> and additions?
18:23:30 <SergeyLukjanov> oh, hext topic is already here
18:23:30 <SergeyLukjanov> #topic Icehouse release status
18:23:37 <SergeyLukjanov> next*
18:24:28 <aignatov> hehe, probably we already discussed Icehouse release status :)
18:24:39 <SergeyLukjanov> #info both action items re aggregating list of needed CRs completed
18:24:54 <SergeyLukjanov> #info done: aignatov to compost list of issues that should be fixed in Icehouse
18:25:00 <SergeyLukjanov> #info done: elmiko to compose list of docs that should be done in Icehouse
18:25:08 <SergeyLukjanov> aignatov, elmiko, thanks
18:25:18 <aignatov> #link https://etherpad.openstack.org/p/icehouse-fixes-after-rc
18:25:24 <SergeyLukjanov> mattf, socialization, you've requested :)
18:25:29 <mattf> will someone explain the functional impact of https://bugs.launchpad.net/sahara/+bug/1304995 ?
18:26:04 <SergeyLukjanov> sreshetnyak, ^^
18:26:05 <aignatov> performance, hadoop can work more faster if it has native libraries
18:26:07 <mattf> what will people be able to see before and after the "fix"?
18:26:32 <SergeyLukjanov> mattf, IIRC it's a significant perf. improvement for hadoop 2 users
18:26:38 <mattf> aignatov, i get that in theory, but has anyone demonstrated it? what kind of performance increase are you looking at...?
18:27:10 <mattf> SergeyLukjanov, if you have some data on that please pls pls toss it in the bug so we can work from the same facts
18:27:13 <aignatov> mattf: yes, try to create cluster w/o those libs and w/ it
18:27:34 <dmitryme> mattf: mapreduce speed up, I think
18:27:44 <mattf> generally i'm +1 for anything potentially perf related, but i'm more interested right now in making sure we communicate what the impact of changes is
18:27:45 <SergeyLukjanov> more info - http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-common/NativeLibraries.html
18:28:19 <aignatov> in the first case you'll see multiple warnings about absence of those libs and all hadoop related actions will work more slowly in comparison with case w/ libs
18:28:55 <SergeyLukjanov> AFAIK hadoop expect that these native libs will be available, it's just weird that they are not included
18:28:58 <aignatov> hadoop 1.2 contained native libs in del and rpm packages so we didn't worried about that
18:29:05 <SergeyLukjanov> correction: only 32-bit ones included
18:29:07 <aignatov> del -> deb
18:29:54 <mattf> imho removing a warning isn't important enough
18:30:45 <mattf> i get that there's a discrepancy between what may be expected and what's provided, but if that doesn't change the user experience significantly why add it now instead of juno?
18:31:18 <dmitryme> mattf: we actually have a backup plan in case images with native libs start to fail
18:31:59 <mattf> dmitryme, you're way ahead of me. so what happens when those libs don't load on fedora because they were built for ubuntu etc?
18:32:19 * mattf grants ^^ is a theoretical issue atm
18:32:33 <dmitryme> we will simply rebuild them without native libs and place new images instead of old ones on our download site
18:32:53 <dmitryme> without changing images' names
18:33:00 <dmitryme> so we will not have to change the docs
18:33:01 * mattf eeks a bit
18:33:14 * mattf expects tosky just shed a tear
18:33:28 <tosky> eh :(
18:33:46 <dmitryme> tosky: why?
18:33:56 <tosky> well, at least the assumption is that "it works as before"
18:34:10 <tosky> so not the top issue - I have other issues at the moment
18:34:34 <tosky> should I ask for info on a bug now, or should I wait for the "open discussion" ?
18:34:46 <tosky> (potentially rc bug if confirmed, I would say)
18:34:53 <SergeyLukjanov> tosky, now
18:34:54 <mattf> tosky, i don't think there's a bug for what to do if native libs fail
18:35:30 <tosky> so, this authentication bug when launching a job https://bugs.launchpad.net/sahara/+bug/1305210 was reported by me but found almost at the same time by elmiko too
18:35:54 <elmiko> on 2 different systems no less
18:35:55 <tosky> he went into details more than me
18:35:57 <mattf> aignatov, dmitryme, SergeyLukjanov, imho the bar for changes during a stabilization period should have a clear functional justification and a way to test it
18:36:35 <sreshetnyak> libhdfs.so library provide C API for hdfs
18:36:38 <mattf> it's probably a bunch of work to do for this native library change, so i'd rather it go in at the beginning of the next cycle so we have 5mo to stumble on the issues
18:36:59 <aignatov> tosky: elmiko that's looks strange because we have savanna-ci which runs Pig jobs on the near-master devstack with keystone v3
18:37:00 <mattf> so i'll probably not +2 it
18:37:08 <SergeyLukjanov> #info RC2 will be earlier friday after merging and backporting last changes
18:37:23 <mattf> this is the kind of criteria that we should lay out for stabilization
18:37:32 <elmiko> aignatov: do we generate the request for the "/v3/tokens" endpoint?
18:37:42 * mattf attempts to step down off soapbox
18:37:56 <SergeyLukjanov> #info only fixes for super critical issues will be accepted after RC2 (like sahara couldn't start because....)
18:37:58 <tosky> aignatov: do you use the internal storage for the jobs, or swift?
18:39:28 <aignatov> hmm
18:39:44 <SergeyLukjanov> mattf, it's only related to the vanilla hadoop 2, that's why we're considering i
18:39:46 <SergeyLukjanov> it*
18:39:49 <aignatov> for jobs I've used internal storage
18:40:00 <SergeyLukjanov> tosky, I think swift is used in CI
18:40:26 <tosky> uhm, could you please check? Also, if I missed some details in the bug, please tell me and I will add the missing steps
18:40:33 <mattf> SergeyLukjanov, that "hadoop 2" bucket is very wide. it would definitely help if someone wrote down in a bp what it meant. but i'm off my soapbox now.
18:40:59 <elmiko> aignatov, SergeyLukjanov, i'm still curious if we are generating the url for the "/v3/tokens" endpoint?
18:41:15 <SergeyLukjanov> elmiko, we're generating it if it's not available
18:41:17 <dmitryme> tosky: the stack trace indicates that the code fails in 'upload_job_files'
18:41:38 <SergeyLukjanov> elmiko, but it's used only for trusts-related stuff
18:41:42 <dmitryme> AFAIU this code is should common for both Hadoop 1 and Hadoop 2
18:41:55 <dmitryme> tmckay, aignatov, can you confirm?
18:42:09 <elmiko> SergeyLukjanov: my concern is that it's not the proper endpoint to keystone
18:42:17 <aignatov> dmitryme: uploading job files is the same for both versions
18:42:24 <elmiko> maybe i'm misunderstanding how we use it
18:43:10 <SergeyLukjanov> https://github.com/openstack/sahara/blob/master/sahara/utils/openstack/base.py#L73
18:43:20 <tmckay> aignatov, dmitryme, agree
18:44:23 <elmiko> SergeyLukjanov: we might have a small issue then, v3 keystone api has "/v3/auth/tokens" but no "/v3/tokens"
18:44:35 <aignatov> elmiko, one more thing could be is the ".sahara" suffix
18:44:49 <aignatov> it is only needed for data sources
18:44:57 <elmiko> aignatov: on the swift:// uri?
18:45:09 <SergeyLukjanov> elmiko, we're not generating the url, we're using keystone client
18:45:34 <aignatov> elmiko: for uploaded job binaries you don't need to provide that suffix
18:45:43 <tmckay> all, actually, I just got this error 10 minutes ago :)
18:45:52 <elmiko> SergeyLukjanov: ok, i just saw the same bug as tosky and somewhere a GET is made on "/v3/tokens" and it returns 404
18:46:08 <SergeyLukjanov> elmiko, probably, old keystoneclient version?
18:46:10 <elmiko> aignatov: ok, i had to remove it for my stuff
18:46:18 <tmckay> using neutron, hadoop 2.3.0, tip of devstack.  But, I only used swift for a binary -- which is not accessed from the hadoop cluster
18:46:21 <elmiko> SergeyLukjanov: could be
18:46:26 <tmckay> it's pulled by sahara and then copied
18:46:40 <tmckay> so, I wonder if it's a problem related to neutron or the tip of devstack
18:46:57 <SergeyLukjanov> elmiko, the -transient job @ sahara-ci tests all our code that is related to keystone api v3
18:47:21 <SergeyLukjanov> elmiko, it creates transient cluster, that means it creates trusts and etc.
18:47:43 <elmiko> SergeyLukjanov: ok, it was just odd that tosky and i saw this on 2 radically different setups
18:48:02 <aignatov> SergeyLukjanov: but transient cluster job doesn't run edp actions
18:48:12 <elmiko> SergeyLukjanov: he was using devstack and i was using rdo/icehouse
18:48:15 <SergeyLukjanov> aignatov, yup, that's correct
18:48:23 <tmckay> elmiko, I should poke at this some, too.  seems to be widespread
18:48:45 <tosky> SergeyLukjanov: transient cluster, do you mean the cluster is created on-the-fly ("Launch on a new cluster")?
18:48:53 <elmiko> tmckay: cool, i'm trying to get my stack working again to see if it's still happening. i ended up having to back off v3 auth to get it working
18:48:58 <SergeyLukjanov> tosky, yup
18:49:06 <tosky> SergeyLukjanov: I ran the job on an existing cluster
18:49:15 <elmiko> tosky: same for me
18:49:21 <SergeyLukjanov> bah
18:49:44 <SergeyLukjanov> which keystone version you have in endpoints?
18:50:19 <tmckay> elmiko, I've got this issue right now :)  I can hit "swift download blah" from the cli, with the same credentials, but not from sahara ...
18:50:53 <tosky> SergeyLukjanov: which file should I check exactly?
18:51:00 <elmiko> SergeyLukjanov: i'm not sure, i think it was v3 but i had to bring down that stack
18:51:12 <dmitryme> tmckay: CLI works, but Sahara does not, with the same credentials, right?
18:51:17 <tmckay> yes
18:51:48 <dmitryme> tmckay: can you show OS_AUTH_URL  you use for CLI?
18:52:07 <SergeyLukjanov> let's continue discussing it after the meeting
18:52:12 <tosky> oki
18:52:25 <SergeyLukjanov> and consider it as a potential issue for rc3
18:52:25 <dmitryme> SergeyLukjanov: right, we can do it in Sahara channel
18:52:48 <SergeyLukjanov> in fact the v2 keystone api version is still the main one
18:53:07 <SergeyLukjanov> and there is still no good deprecation plan for it
18:53:18 <SergeyLukjanov> #topic Open discussion
18:53:19 <tosky> didn't they switch it already? Someone was even talking about killing the tests in Juno timeframe
18:53:56 <SergeyLukjanov> tosky, they was stopped by tc due to the attempt to depricate v2 api while not all other projects support v3 :)
18:54:13 <SergeyLukjanov> 6 mins left to discuss something
18:54:41 <mattf> has anyone filed a bp for massaging itests to look like tempest tests?
18:54:47 <elmiko> i have been using sahara-image-elements to create centos+hdp2 images, i continue to see ambari-server setup failuers during boot.
18:55:12 <mattf> elmiko, no hwx folks around, doh
18:55:21 <elmiko> oh well, i tried
18:55:28 <dmitryme> Hey people, I wanted to remind you that we have https://ask.openstack.org/en/questions/
18:55:36 <aignatov> elmiko: I'm afraid no one except HW knows how HDP 2 plugin works
18:55:44 <SergeyLukjanov> ylobankov_, please, ensure "[22:54:42]  <mattf>	 has anyone filed a bp for massaging itests to look like tempest tests?" is done
18:55:48 <dmitryme> where people post questions for Sahara (and Savanna) from time to time
18:56:07 <elmiko> aignatov: i've made it work, but only if i give the instance access to the internet. i'll ask them next time we talk.
18:56:08 <mattf> ylobankov_, please subscribe me when it's done
18:56:58 <aignatov> ok
18:57:21 <dmitryme> elmiko: if installer does not find JRE or HDP packages on the image, it starts downloading them from Hortonworks mirror
18:57:34 <ylobankov_> mattf: ok
18:57:57 <elmiko> dmitryme: yea, according to the hwx guys they were installed
18:58:26 <dmitryme> elmiko: maybe different version of packages?
18:59:46 <aignatov> let's move to the #openstack-sahara channel
18:59:53 <mattf> thanks folks!
18:59:56 <SergeyLukjanov> thanks all!
18:59:59 <SergeyLukjanov> #endmeeting