Val and def
In the above class definition ‘val’ means, that the class has the fields head and tail. Without ‘val’ those fields are private. So ‘val’ is a way to define a […]
Scala
Programming
Coin change problem
If somebody asks you to change a 10 cent coin and you have an unlimited number of 1 cent coins, 2 cent coins, and 5 cent coins, in how many
Programming, Scala
Call by Name vs Call by Value in Scala
This information is based on the coursera course “Functional Programming Principles in Scala” by Martin Odersky. In Scala expressions* are evaluated by the Scala interpreter following the substitution model. What
Spark
Spark Streaming
Here is an example of how to do a very simple streaming from and to file with pyspark. https://gist.github.com/sallos-cyber/a14e03da49cc0c873a651628dba4d096
Frontend
How to serve local web files over HTTP for testing
Recently I got interested in creating my own webapp. If you write a simple html file you can open it with a browser. Notice that in the browser in the
Spark
SPARK configuration
spark.dynamicAllocation.enabled spark.dynamicAllocation.initialExecutors spark.dynamicAllocation.minExecutors spark.dynamicAllocation.maxExecutors spark.shuffle.partitions spark.default.parallelism = spark.executor.instances * spark.executor.cores * 2 maxPartitionBytes Input bytes = 40 GB? Wähle so viele Partitions, so dass die Größe einer Partition <= 200