All Downloads are FREE. Search and download functionalities are using the official Maven repository.

Download big-map_2.10 JAR file with all dependencies

Search JAR files by class name

big-map_2.10 from group com.avast (version 1.1)

In some data processing tasks we need to use huge maps or sets that are bigger than available JVM heap space or they are loading too slow to standard Java or Scala Maps. We use TSV format (text file with tab separated columns) for persist this kind of Maps or Sets. Some columns are used as a key and rest of columns as a value. Idea of this library is simple. We can prepare these maps once (sort by key), store it to file and then use it as memory mapped file. Searching key in sorted file has log(n) complexity. If more processes uses the same memory mapped file, it exists in memory just once (on Linux). This file can be loaded lazy by OS.

Group: com.avast Artifact: big-map_2.10
Show documentation Show source 
 

0 downloads
Artifact big-map_2.10
Group com.avast
Version 1.1
Last update 24. October 2016
Tags: need they store columns persist same used heap more file value sets processing slow these processes text once searching loading prepare available format huge loaded that than then rest some this mapped idea sort bigger simple complexity data separated space scala kind just with standard sorted tasks linux memory maps library exists java uses lazy
Organization AVAST Software
URL https://github.com/avast/BigMap
License Apache License, Version 2.0
Dependencies amount 5
Dependencies scala-library, scopt_${build.scopt.scala.version}, slf4j-api, logback-classic, commons-io,
There are maybe transitive dependencies!



Page 1 from 1 (items total 1)


© 2015 - 2024 Weber Informatics LLC | Privacy Policy