18:05:19 <SergeyLukjanov> #startmeeting sahara 18:05:20 <openstack> Meeting started Thu Apr 10 18:05:19 2014 UTC and is due to finish in 60 minutes. The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:05:21 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:05:24 <openstack> The meeting name has been set to 'sahara' 18:05:27 <SergeyLukjanov> #ling https://wiki.openstack.org/wiki/Meetings/SaharaAgenda 18:05:34 <SergeyLukjanov> hey folks 18:05:49 <SergeyLukjanov> traditionally, the first topic is.... 18:05:51 <SergeyLukjanov> #topic News / updates 18:05:55 <SergeyLukjanov> folks, please 18:06:25 <dmitryme> I was mostly finalizing the docs 18:06:30 <crobertsrh> I've started the merging process for the dashboard into horizon. I hope to have all the CRs up no later than tomorrow. 18:06:40 <mattf> sahara is going to get air time during #devnation next week (assuming i start & finish the presentation). also, it'll get air time during hadoop summit in san jose. 18:07:07 <ylobankov_> I am testing all things related with Sahara 18:07:12 <aignatov> mattf: cool! 18:07:53 <mattf> aignatov, i may snag your todo demo for #devnation. i can't seem to get my sigmask in place to make something. 18:08:25 <aignatov> I was working a few on docs and tested all related heat stuff in sahara, also updated sahara's resource in heat 18:08:30 <mattf> re conference - has anyone submitted to hadoop world in nyc? 18:08:53 <aignatov> mattf: you mean demo about "Top todoers"? 18:09:18 <SergeyLukjanov> crobertsrh, awesome, could you please announce new CRs to #openstack-sahara? 18:09:37 <aignatov> feel free to show it for masses :) 18:09:40 <mattf> aignatov, quite possibly 18:09:45 <crobertsrh> #link https://review.openstack.org/#/c/86648/ 18:09:51 <SergeyLukjanov> #link https://etherpad.openstack.org/p/sahara-icehouse-tbd 18:09:58 <crobertsrh> #link https://review.openstack.org/#/c/86614/ 18:10:10 <mattf> aignatov, i'm thinking about adding a time component. who's removing their todos vs who is just accumulating them 18:10:40 <SergeyLukjanov> #info the blocker for RC2 is HDP features / desciption update https://review.openstack.org/#/c/84797/ 18:10:49 <SergeyLukjanov> when it'll be merged, we're reading to rc2 18:10:58 <dmitryme> mattf: sounds like an interesting variation 18:11:05 <SergeyLukjanov> I'll double check that all needed CRs are backported to m-p 18:11:11 <mattf> dmitryme, i'll look pretty bad in it 18:11:18 <SergeyLukjanov> and ask ttx to push the rc2 tag 18:11:24 <mattf> dmitryme, could be good for public shaming 18:11:36 <aignatov> mattf: nice idea, I mean to get sources for one date and sources for another date and then a diff in TODOs, right? 18:11:59 <dmitryme> SergeyLukjanov: can we put https://review.openstack.org/#/c/86641/ into rc2 as well? 18:12:01 <mattf> aignatov, for speed i might just pull release tarballs 18:12:05 <tmckay> all, I'm pursuing the bigpetstore app as a demo, but it's not there yet. 2 parts, data generation, and csv processing. I ran the first part (with hdfs, needs mods to use swift). The 2nd part launches PigServer from inside a java app, so I'm trying to run it as a Java action but it needs a pig lib on the classpath. The other option is to extract the Pig script built in the java app and allow Oozie's pig runner to work on that. 18:12:22 <aignatov> probably this idea is a good on more demo's candidate for Atalanta summit 18:13:10 <mattf> ideally we can have a different demo for #devnation, os summit, hadoop summit 18:13:11 <SergeyLukjanov> dmitryme, sure, nice tip 18:13:39 <SergeyLukjanov> mattf, aignatov, jspeidel, alazarev, please, review/approve https://review.openstack.org/#/c/86641/ 18:13:56 <mattf> SergeyLukjanov, i've been super distracted, but at some point i'd like to formalize the criteria for breaching feature freeze or rc 18:14:38 <SergeyLukjanov> mattf, it was already defined - hadoop2 updates and docs 18:14:59 <sreshetnyak> o/ 18:15:25 <mattf> SergeyLukjanov, my sampling of merges suggests we're not sticking to that. your notion of the criteria must not match my notion. 18:15:27 <aignatov> yep, seems to be hadoop 2 related changes are already in the icehouse 18:15:34 <aignatov> only docs left 18:15:46 <SergeyLukjanov> mattf, and open issue, that are blocking some important features 18:15:50 <SergeyLukjanov> #info https://launchpad.net/sahara/+milestone/icehouse-rc2 18:16:16 <mattf> important is subjective 18:16:39 <SergeyLukjanov> mattf, note that master is Juno already, not everything merged to master backported to m-p 18:17:14 <mattf> SergeyLukjanov, /me nods 18:17:50 <mattf> i'd like to discuss at some point, but after next week... 18:18:18 <aignatov> release will be on the next week :) 18:18:26 * mattf nods 18:18:32 <SergeyLukjanov> mattf, we'll have much more for this process next release 18:18:43 * mattf breaks neck nodding 18:19:19 <SergeyLukjanov> mattf, this time we have last two weeks before FF renaming savanna to sahara instead of doing things that we're doing now 18:19:49 <SergeyLukjanov> so, that's why I've approved more backports than I wnat to see during the rc perion 18:19:50 <mattf> SergeyLukjanov, i'm just raising a warning flag that we should find a way to formalize the criteria 18:19:53 <SergeyLukjanov> period* 18:19:58 <mattf> and clearly it's too late for this release 18:20:18 <SergeyLukjanov> mattf, there is no way to formalize it ;) take a look on other projects 18:20:35 <mattf> well, at least socialize it then 18:20:50 <mattf> but...we should move on 18:21:13 <SergeyLukjanov> all of the stuff was added to etherpad and milestone that were announced and discussed several times in irc and ml 18:21:41 <SergeyLukjanov> oh, I find useful link 18:21:44 <SergeyLukjanov> #link https://github.com/openstack/sahara/compare/2014.1.rc1...milestone-proposed 18:21:50 <mattf> i really don't want to entirely derail the meeting w/ details 18:22:57 <SergeyLukjanov> so, for rc2, we're waiting for https://review.openstack.org/#/c/86641/ https://review.openstack.org/#/c/84797/ and https://review.openstack.org/#/c/83036/ 18:23:06 <SergeyLukjanov> any objections? 18:23:11 <SergeyLukjanov> and additions? 18:23:30 <SergeyLukjanov> oh, hext topic is already here 18:23:30 <SergeyLukjanov> #topic Icehouse release status 18:23:37 <SergeyLukjanov> next* 18:24:28 <aignatov> hehe, probably we already discussed Icehouse release status :) 18:24:39 <SergeyLukjanov> #info both action items re aggregating list of needed CRs completed 18:24:54 <SergeyLukjanov> #info done: aignatov to compost list of issues that should be fixed in Icehouse 18:25:00 <SergeyLukjanov> #info done: elmiko to compose list of docs that should be done in Icehouse 18:25:08 <SergeyLukjanov> aignatov, elmiko, thanks 18:25:18 <aignatov> #link https://etherpad.openstack.org/p/icehouse-fixes-after-rc 18:25:24 <SergeyLukjanov> mattf, socialization, you've requested :) 18:25:29 <mattf> will someone explain the functional impact of https://bugs.launchpad.net/sahara/+bug/1304995 ? 18:26:04 <SergeyLukjanov> sreshetnyak, ^^ 18:26:05 <aignatov> performance, hadoop can work more faster if it has native libraries 18:26:07 <mattf> what will people be able to see before and after the "fix"? 18:26:32 <SergeyLukjanov> mattf, IIRC it's a significant perf. improvement for hadoop 2 users 18:26:38 <mattf> aignatov, i get that in theory, but has anyone demonstrated it? what kind of performance increase are you looking at...? 18:27:10 <mattf> SergeyLukjanov, if you have some data on that please pls pls toss it in the bug so we can work from the same facts 18:27:13 <aignatov> mattf: yes, try to create cluster w/o those libs and w/ it 18:27:34 <dmitryme> mattf: mapreduce speed up, I think 18:27:44 <mattf> generally i'm +1 for anything potentially perf related, but i'm more interested right now in making sure we communicate what the impact of changes is 18:27:45 <SergeyLukjanov> more info - http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-common/NativeLibraries.html 18:28:19 <aignatov> in the first case you'll see multiple warnings about absence of those libs and all hadoop related actions will work more slowly in comparison with case w/ libs 18:28:55 <SergeyLukjanov> AFAIK hadoop expect that these native libs will be available, it's just weird that they are not included 18:28:58 <aignatov> hadoop 1.2 contained native libs in del and rpm packages so we didn't worried about that 18:29:05 <SergeyLukjanov> correction: only 32-bit ones included 18:29:07 <aignatov> del -> deb 18:29:54 <mattf> imho removing a warning isn't important enough 18:30:45 <mattf> i get that there's a discrepancy between what may be expected and what's provided, but if that doesn't change the user experience significantly why add it now instead of juno? 18:31:18 <dmitryme> mattf: we actually have a backup plan in case images with native libs start to fail 18:31:59 <mattf> dmitryme, you're way ahead of me. so what happens when those libs don't load on fedora because they were built for ubuntu etc? 18:32:19 * mattf grants ^^ is a theoretical issue atm 18:32:33 <dmitryme> we will simply rebuild them without native libs and place new images instead of old ones on our download site 18:32:53 <dmitryme> without changing images' names 18:33:00 <dmitryme> so we will not have to change the docs 18:33:01 * mattf eeks a bit 18:33:14 * mattf expects tosky just shed a tear 18:33:28 <tosky> eh :( 18:33:46 <dmitryme> tosky: why? 18:33:56 <tosky> well, at least the assumption is that "it works as before" 18:34:10 <tosky> so not the top issue - I have other issues at the moment 18:34:34 <tosky> should I ask for info on a bug now, or should I wait for the "open discussion" ? 18:34:46 <tosky> (potentially rc bug if confirmed, I would say) 18:34:53 <SergeyLukjanov> tosky, now 18:34:54 <mattf> tosky, i don't think there's a bug for what to do if native libs fail 18:35:30 <tosky> so, this authentication bug when launching a job https://bugs.launchpad.net/sahara/+bug/1305210 was reported by me but found almost at the same time by elmiko too 18:35:54 <elmiko> on 2 different systems no less 18:35:55 <tosky> he went into details more than me 18:35:57 <mattf> aignatov, dmitryme, SergeyLukjanov, imho the bar for changes during a stabilization period should have a clear functional justification and a way to test it 18:36:35 <sreshetnyak> libhdfs.so library provide C API for hdfs 18:36:38 <mattf> it's probably a bunch of work to do for this native library change, so i'd rather it go in at the beginning of the next cycle so we have 5mo to stumble on the issues 18:36:59 <aignatov> tosky: elmiko that's looks strange because we have savanna-ci which runs Pig jobs on the near-master devstack with keystone v3 18:37:00 <mattf> so i'll probably not +2 it 18:37:08 <SergeyLukjanov> #info RC2 will be earlier friday after merging and backporting last changes 18:37:23 <mattf> this is the kind of criteria that we should lay out for stabilization 18:37:32 <elmiko> aignatov: do we generate the request for the "/v3/tokens" endpoint? 18:37:42 * mattf attempts to step down off soapbox 18:37:56 <SergeyLukjanov> #info only fixes for super critical issues will be accepted after RC2 (like sahara couldn't start because....) 18:37:58 <tosky> aignatov: do you use the internal storage for the jobs, or swift? 18:39:28 <aignatov> hmm 18:39:44 <SergeyLukjanov> mattf, it's only related to the vanilla hadoop 2, that's why we're considering i 18:39:46 <SergeyLukjanov> it* 18:39:49 <aignatov> for jobs I've used internal storage 18:40:00 <SergeyLukjanov> tosky, I think swift is used in CI 18:40:26 <tosky> uhm, could you please check? Also, if I missed some details in the bug, please tell me and I will add the missing steps 18:40:33 <mattf> SergeyLukjanov, that "hadoop 2" bucket is very wide. it would definitely help if someone wrote down in a bp what it meant. but i'm off my soapbox now. 18:40:59 <elmiko> aignatov, SergeyLukjanov, i'm still curious if we are generating the url for the "/v3/tokens" endpoint? 18:41:15 <SergeyLukjanov> elmiko, we're generating it if it's not available 18:41:17 <dmitryme> tosky: the stack trace indicates that the code fails in 'upload_job_files' 18:41:38 <SergeyLukjanov> elmiko, but it's used only for trusts-related stuff 18:41:42 <dmitryme> AFAIU this code is should common for both Hadoop 1 and Hadoop 2 18:41:55 <dmitryme> tmckay, aignatov, can you confirm? 18:42:09 <elmiko> SergeyLukjanov: my concern is that it's not the proper endpoint to keystone 18:42:17 <aignatov> dmitryme: uploading job files is the same for both versions 18:42:24 <elmiko> maybe i'm misunderstanding how we use it 18:43:10 <SergeyLukjanov> https://github.com/openstack/sahara/blob/master/sahara/utils/openstack/base.py#L73 18:43:20 <tmckay> aignatov, dmitryme, agree 18:44:23 <elmiko> SergeyLukjanov: we might have a small issue then, v3 keystone api has "/v3/auth/tokens" but no "/v3/tokens" 18:44:35 <aignatov> elmiko, one more thing could be is the ".sahara" suffix 18:44:49 <aignatov> it is only needed for data sources 18:44:57 <elmiko> aignatov: on the swift:// uri? 18:45:09 <SergeyLukjanov> elmiko, we're not generating the url, we're using keystone client 18:45:34 <aignatov> elmiko: for uploaded job binaries you don't need to provide that suffix 18:45:43 <tmckay> all, actually, I just got this error 10 minutes ago :) 18:45:52 <elmiko> SergeyLukjanov: ok, i just saw the same bug as tosky and somewhere a GET is made on "/v3/tokens" and it returns 404 18:46:08 <SergeyLukjanov> elmiko, probably, old keystoneclient version? 18:46:10 <elmiko> aignatov: ok, i had to remove it for my stuff 18:46:18 <tmckay> using neutron, hadoop 2.3.0, tip of devstack. But, I only used swift for a binary -- which is not accessed from the hadoop cluster 18:46:21 <elmiko> SergeyLukjanov: could be 18:46:26 <tmckay> it's pulled by sahara and then copied 18:46:40 <tmckay> so, I wonder if it's a problem related to neutron or the tip of devstack 18:46:57 <SergeyLukjanov> elmiko, the -transient job @ sahara-ci tests all our code that is related to keystone api v3 18:47:21 <SergeyLukjanov> elmiko, it creates transient cluster, that means it creates trusts and etc. 18:47:43 <elmiko> SergeyLukjanov: ok, it was just odd that tosky and i saw this on 2 radically different setups 18:48:02 <aignatov> SergeyLukjanov: but transient cluster job doesn't run edp actions 18:48:12 <elmiko> SergeyLukjanov: he was using devstack and i was using rdo/icehouse 18:48:15 <SergeyLukjanov> aignatov, yup, that's correct 18:48:23 <tmckay> elmiko, I should poke at this some, too. seems to be widespread 18:48:45 <tosky> SergeyLukjanov: transient cluster, do you mean the cluster is created on-the-fly ("Launch on a new cluster")? 18:48:53 <elmiko> tmckay: cool, i'm trying to get my stack working again to see if it's still happening. i ended up having to back off v3 auth to get it working 18:48:58 <SergeyLukjanov> tosky, yup 18:49:06 <tosky> SergeyLukjanov: I ran the job on an existing cluster 18:49:15 <elmiko> tosky: same for me 18:49:21 <SergeyLukjanov> bah 18:49:44 <SergeyLukjanov> which keystone version you have in endpoints? 18:50:19 <tmckay> elmiko, I've got this issue right now :) I can hit "swift download blah" from the cli, with the same credentials, but not from sahara ... 18:50:53 <tosky> SergeyLukjanov: which file should I check exactly? 18:51:00 <elmiko> SergeyLukjanov: i'm not sure, i think it was v3 but i had to bring down that stack 18:51:12 <dmitryme> tmckay: CLI works, but Sahara does not, with the same credentials, right? 18:51:17 <tmckay> yes 18:51:48 <dmitryme> tmckay: can you show OS_AUTH_URL you use for CLI? 18:52:07 <SergeyLukjanov> let's continue discussing it after the meeting 18:52:12 <tosky> oki 18:52:25 <SergeyLukjanov> and consider it as a potential issue for rc3 18:52:25 <dmitryme> SergeyLukjanov: right, we can do it in Sahara channel 18:52:48 <SergeyLukjanov> in fact the v2 keystone api version is still the main one 18:53:07 <SergeyLukjanov> and there is still no good deprecation plan for it 18:53:18 <SergeyLukjanov> #topic Open discussion 18:53:19 <tosky> didn't they switch it already? Someone was even talking about killing the tests in Juno timeframe 18:53:56 <SergeyLukjanov> tosky, they was stopped by tc due to the attempt to depricate v2 api while not all other projects support v3 :) 18:54:13 <SergeyLukjanov> 6 mins left to discuss something 18:54:41 <mattf> has anyone filed a bp for massaging itests to look like tempest tests? 18:54:47 <elmiko> i have been using sahara-image-elements to create centos+hdp2 images, i continue to see ambari-server setup failuers during boot. 18:55:12 <mattf> elmiko, no hwx folks around, doh 18:55:21 <elmiko> oh well, i tried 18:55:28 <dmitryme> Hey people, I wanted to remind you that we have https://ask.openstack.org/en/questions/ 18:55:36 <aignatov> elmiko: I'm afraid no one except HW knows how HDP 2 plugin works 18:55:44 <SergeyLukjanov> ylobankov_, please, ensure "[22:54:42] <mattf> has anyone filed a bp for massaging itests to look like tempest tests?" is done 18:55:48 <dmitryme> where people post questions for Sahara (and Savanna) from time to time 18:56:07 <elmiko> aignatov: i've made it work, but only if i give the instance access to the internet. i'll ask them next time we talk. 18:56:08 <mattf> ylobankov_, please subscribe me when it's done 18:56:58 <aignatov> ok 18:57:21 <dmitryme> elmiko: if installer does not find JRE or HDP packages on the image, it starts downloading them from Hortonworks mirror 18:57:34 <ylobankov_> mattf: ok 18:57:57 <elmiko> dmitryme: yea, according to the hwx guys they were installed 18:58:26 <dmitryme> elmiko: maybe different version of packages? 18:59:46 <aignatov> let's move to the #openstack-sahara channel 18:59:53 <mattf> thanks folks! 18:59:56 <SergeyLukjanov> thanks all! 18:59:59 <SergeyLukjanov> #endmeeting