This information is based on the coursera course “Functional Programming Principles in Scala” by Martin Odersky. In Scala expressions* are evaluated by the Scala interpreter following the substitution model. What […]
Here is an example of how to do a very simple streaming from and to file with pyspark. https://gist.github.com/sallos-cyber/a14e03da49cc0c873a651628dba4d096
Recently I got interested in creating my own webapp. If you write a simple html file you can open it with a browser. Notice that in the browser in the
spark.dynamicAllocation.enabled spark.dynamicAllocation.initialExecutors spark.dynamicAllocation.minExecutors spark.dynamicAllocation.maxExecutors spark.shuffle.partitions spark.default.parallelism = spark.executor.instances * spark.executor.cores * 2 maxPartitionBytes Input bytes = 40 GB? Wähle so viele Partitions, so dass die Größe einer Partition <= 200
A random variable $Z$ is said to have the standard normal distribution, if its probability density function (pdf) is as follows: \[\begin{equation}f_Z(z)=\frac{1}{\sqrt{2\pi}} * \exp(\frac{-z^2}{2}), \quad -\infty<z<\infty\end{equation} \tag{1}\label{eq:eq1} \] This formula
https://gist.github.com/sallos-cyber/d2bc782f09688beefb4e66068af03174
