java - Spark sql with hive table delta support -


as latest version of spark (2.1.0) lists unsupported major functionality of hive like:

tables buckets: bucket hash partitioning within hive table partition. spark sql doesn’t support buckets yet.

it means hive tables have bucketed column structure won't loaded dataframe processed properly. there workaround achieve such functionality via jdbc, temp tables , etc. main question how achieve full control using spark sql on tables materialized deltas ?


Comments

Post a Comment

Popular posts from this blog

java - How to handle special characters while unmarshalling xml in JAXB -

my test xml content <p id="033" num="03">geopotent change&#x2 high.</p> and run jaxb unmarshalling, i'm getting exception 09:58:43.748 error [main][net.serviceimpl] parsing error: javax.xml.bind.unmarshalexception- linked exception: [javax.xml.stream.xmlstreamexception: parseerror @ [row,col]: [161,306]message: string "&#] my jaxb unmarshal source jaxbcontext jaxbcontext = jaxbcontext.newinstance(cndocument.class); unmarshaller jaxbunmarshaller = jaxbcontext.createunmarshaller(); document = (cndocument) jaxbunmarshaller.unmarshal(xmlfile); how can escape characters? (&#x2) you jaxb parse error because xml content not well-formed. should &#x2; (with semicolon), not &#x2 .
Read more

ios - Pass NSDictionary from Javascript to Objective-c in JavascriptCore -

i'm using javascriptcore in app. now, i'm passing variables jscontext can passed objective-c. however, 1 of variables, nsdictionary, not passing through correctly. run code below: var evaluate = function(variables) { app.setdictionary(variables.dictionary); } this simple example has these following methods set in jscontext . this setdictionary() method: - (void)setdicationary:(nsdictionary *)dictionary { self.mutabledictionary = [dictionary mutablecopy]; } this variables.dictionary : - (nsdictionary *)dictionary { return self.values; } and how call evaluate() : jsvalue *jsfunction = self.context[@"evaluate"]; jsvalue *value = [jsfunction callwitharguments:@[self.variables]]; however, in setdictionary method, don't nsdictionary, instead nsstring containing [object object] . any ideas how can solve this? although javascriptcore automatically converts types between objective-c or swift , javascript, suggest implement ex...
Read more

tensorflow when input_data MNIST_data , zlib.error: Error -3 while decompressing: invalid block type -

i'm geting started tensorflow,followed tutorials,but cannot download mnist data. import tensorflow tf tensorflow.examples.tutorials.mnist import input_data mnist=input_data.read_data_sets("mnist_data/",one_hot=true) print("download done!") error information: extracting mnist_data/train-images-idx3-ubyte.gz traceback (most recent call last): file "/home/sj/program/tesorflow_try.py", line 6, in <module> mnist=input_data.read_data_sets("mnist_data/",one_hot=true) file "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py", line 213, in read_data_sets train_images = extract_images(f) file "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py", line 60, in extract_images buf = bytestream.read(rows * cols * num_images) file "/usr/lib/python2.7/gzip.py", line 268, in read self._read(readsize) file...
Read more