• Home
• io.eels
• eel-hive_2.11
• 1.3.0-a13
• source code
• PartitionStrategy.scala

×

Project download

Many resources are needed to download a project. Please understand that we have to compensate our server costs. Thank you in advance.
Project price only 1 $

You can buy this project and download/modify it how often you want.

io.eels.component.hive.partition.PartitionStrategy.scala Maven / Gradle / Ivy

package io.eels.component.hive.partition

import io.eels.schema.Partition
import org.apache.hadoop.fs.{FileSystem, Path}
import org.apache.hadoop.hive.metastore.IMetaStoreClient

/**
  * Strategy that is invoked for each partition. The strategy determines how the metastore is updated
  * to handle partitions. For example, it may choose to create partitions that don't exist, or throw
  * an exception if the partition does not already exist.
  *
  * Note: Any new paths must be created on disk by this strategy.
  *
  * The strategy also determines the location of the path where the partition is located,
  * so custom implementations can place partitions in non standard locations if required.
  *
  * Note: Implementations must be thread safe as multiple writers may invoke
  * the strategy concurrently for the same partition value.
  */
trait PartitionStrategy {
  // for the given partition, the strategy must return the full path for the partition or throw an exception
  def ensurePartition(partition: Partition,
                      dbName: String,
                      tableName: String,
                      inheritPermissions: Boolean,
                      client: IMetaStoreClient)(implicit fs: FileSystem): Path
}