Questions tagged [scala]
Scala is a general purpose programming language principally targeting the Java Virtual Machine. Designed to express common programming patterns in a concise, elegant, and type-safe way, it fuses both imperative and functional programming styles.
240 questions
0
votes
0
answers
86
views
How to connect to SFTP using Apache Spark 3.5 with Scala 2.12 for parallel file transfers?
I am working on a project where I need to transfer thousands of files (each sized between 50-60 MB) every hour from an SFTP server to local storage or AWS S3. I am using Apache Spark 3.5 with Scala 2....
- 11
1
vote
3
answers
508
views
Scala Option vs. conditional branches
If I'm writing Scala functions and have to check for nulls in situations where I can't avoid it (say, working with Spark UDFs and some legacy Java types), is it better to turn things into Option or to ...
- 1,369
0
votes
1
answer
253
views
Sharing akka actors on multiple backend instances
I have a web applications (angular) which is connected to backend using websockets. The role of web app is to display data from multiple sources (which are updated constantly by some Scala Spark apps) ...
- 117
0
votes
1
answer
175
views
Postgres: Storing a task instruction to be run on a schedule, with multiple workers
As a prerequisite I will say that we are wedded to postgres to provide this solution.
This is a very broad problem, and I think it must be solved many times over, but unfortunately as it's so general ...
- 103
0
votes
2
answers
1k
views
Good design for a class with multiple methods to test but one public method
I've been trying to refactor some existing code which is in essence a giant nested procedural call inside what should otherwise be an object oriented architecture. The entry point to the relevant code ...
- 23
0
votes
2
answers
331
views
Trying to understand how this class representation truly represents Natural numbers in Scala
Following Martin Odersky's course on coursera - Functional Programming with Scala and I'm on Week 4 where we're learning about Types and Pattern Matching. In the video lecture, this is the ...
- 157
-2
votes
1
answer
693
views
Export huge excel file
I develop a web application in Angular (frontend) and Scala (backend) for a big data team. Because they use large files for export/import, I build a module which is a copy of Microsoft Excel.
So, what ...
- 117
-3
votes
2
answers
2k
views
What is the benefit of Java collection streams over C# or Scala collections?
Java collection streams were introduced in Java 8, which came out in March of 2014.
By that time, we already had well-established mechanisms for manipulating collections in several other languages, ...
- 32.8k
1
vote
1
answer
230
views
How to avoid code duplication from handling "structually similar types" in Scala?
Often, when programming, you'll have different degrees of information to you in different contexts.
For example, a web server may have two routes, which recieve information about a Person, one of ...
- 174
3
votes
2
answers
265
views
Operate on data that doesn't fit into JVM
We have a service where we have billions of key-value data stored in some storage. Before actually querying the data, we query the bloom filter to determine if the key may exist or definitely does not ...
- 139
1
vote
1
answer
211
views
Is my server design safe regarding multiple threads and concurrent database reads/writes?
I'm making a chat server using sockets and a MySQL database, and after it's working I want to expand it to become a more complex game server.
I want to know whether my design is missing anything.
...
- 111
1
vote
1
answer
456
views
Preferable design for overriding sealed trait method in Scala case classes/objects
Say I have a sealed trait Person that requires the definition of a method work :: Unit -> Unit. I then have three case objects Accountant, Doctor, and Lawyer. I can think of two ways to implement ...
- 121
0
votes
1
answer
358
views
Why is this "more efficient" version of a Scala Collatz solution slower than tail recursion?
I implememted two versions of the collatz problem and felt an icy terror in the pit of my stomach as an optimized solution was slower than tail. The tail recursion is simple:
// calculate the next ...
- 157
3
votes
3
answers
608
views
Does variance make sense in a fully immutable language?
In many OOP programming languages, types can be made co-, contra- or in- variant. Most (if not all of these languages) are able to let variables be mutated in place, i.e. they are not fully immutable ...
- 141
0
votes
2
answers
2k
views
Scala Option apply method when passed a None
Not sure if this is an appropriate question for here, please let me know!
In Scala, the ever so useful Option class has an apply method in its companion object that allows us to quickly wrap any ...
- 17
2
votes
2
answers
1k
views
Is it a good idea to use "lazy val" for correctness?
In Scala, declaring a val as lazy means that its value won't be evaluated until it's used for the first time. This is often explained/demonstrated as being useful for optimization, in case a value ...
2
votes
1
answer
143
views
Extends for feature (aka mix-in) in Scala is bad practice, isn't it?
In Scala, sometimes class extends a trait to provide some methods to extending class inside.
ScalaTest
import org.scalatest._
class ExampleSpec extends FlatSpec with Matchers { ...
Matchers trait ...
- 37
3
votes
3
answers
2k
views
Define multiple DDD events or just one XXXChanged event when using CQRS?
Is it preferable to define a single 'container event' like below:
trait UserStatus
case object Active extends UserStatus
case object Inactive extends UserStatus
case class UserStatusChanged(...
- 39
-1
votes
1
answer
551
views
In which order functions should be defined scala methods
I have some values and methods to define in a scala object.
I do not know in which order I should define them. Is there a "good way" to do it? Such as private methods first then public methods? Only ...
- 101
2
votes
2
answers
451
views
Scala: what has been the desired design consideration for the null instantiation in the example
Or simply asked WHY ?
It simply does not seems logical ...
if ( null.asInstanceOf[String] == null ) println
("null.asIstanceOf[String] is null")
println ("BUT !!!")
if ( null.asInstanceOf[...
- 169
1
vote
2
answers
158
views
How can an iterative algorithm be controlled dynamically?
Suppose we need an iterative algorithm for mathematical optimisation. Each iteration takes a long and random time. After each iteration, a stopping condition is checked for the iterate x, based on ...
- 853
2
votes
1
answer
104
views
Architecture and Database re-design for migration and support of multiple clients
I'm evaluating the migration of the following application's architecture: - Nginx + PHP + MySQL - Currently the infrastructure is scalable and redundant in the AWS cloud and It was designed to support ...
- 131
12
votes
2
answers
5k
views
What's the difference between a Mixin and a Trait?
From what I can tell from Scala and Hack-
Mixins:
Can have state (ie. instance properties)
Can only provide concrete methods
Can have constructors, that are called in the same order that their ...
- 273
4
votes
0
answers
320
views
Should I pass presenter to use case when implement clean archeiture using scala
This question came to me when I was trying implement Clean Architecture using Scala, and come across this post.
In the accepted answer, @candiedorange emphasis on the separation of responsibility, and ...
- 201
1
vote
1
answer
316
views
Building a real-time API on a Kafka/Kinesis-centric structure
I am a beginner to the event-driven data architecture using Kafka / Kinesis as the centrepiece (currently Kinesis) and I have some questions regarding how to build a WebSocket API on such a structure. ...
- 111
2
votes
2
answers
192
views
Trait and Implementation vs just Implementation
In my previous Java job, we tended to avoid defining interfaces for everything unless we genuinely had multiple implementations for it. This may just be my opinion, but if there is only one ...
- 108
1
vote
1
answer
77
views
Java/Maven and Scala/sbt projects share immutable thing
Project 1: a Java/Maven project
Project 2: a Scala/sbt project
Thing: generally, an immutable object instantiated in from 3rd party Java library. For example, ThingBuilder.foo("bar").build()
...
- 736
0
votes
2
answers
134
views
Do Scala worksheet (or REPL logs) belong to the SCM?
The Eclipse Scala IDE (and Intellij Idea, too) has, together with a standard REPL CLI, an artifact named Worksheet, that works like a persistent REPL log: the whole file is compiled and executed at ...
5
votes
1
answer
896
views
Is using Option#get really a bad idea here?
I'm working on a Scala project and Wartremover shows an error about Option#get usage in my code:
Option#get is disabled - use Option#fold instead
While I do understand how get should often be avoided,...
3
votes
1
answer
432
views
Dependencies between functions-only modules: hardcoding vs alternatives
In switching from a procedural background to "FP in the small, OO in the large" I'm grappling with the following problem. Suppose there're modules, each only containing numerical math functions ...
- 853
1
vote
1
answer
1k
views
Does dependency injection fly in the face of functional programming?
I have the following pure function (f2 and f3 are pure too):
class X {
def f1(arg: Type1): Type2 = {
val x = f2(arg)
val y = f3(x)
y
}
def f2...
def f3...
}
Now, I would like ...
- 129
9
votes
2
answers
11k
views
Which approach for integrating Python code into a mainly Scala application did you use a second time because it was successful? [closed]
I have a mainly Scala application and I am interested in approaches to integrating Python code into this application in a way that is proven by you personally to be successful.
In this context
...
- 235
0
votes
1
answer
194
views
Unit testing certain tasks are done in a process
I have some Scala code that I wish to unit test using ScalaMock:
class CancellationManagerSpec extends FlatSpec with MockFactory {
"Cancelling a shipment" should "check shipment is not already ...
- 109
5
votes
1
answer
861
views
Class table inheritance using Slick: always an anti-pattern?
My application's model includes a number of entities that share many common properties, and are operated on by many of the same classes.
If I was using an ORM, I'd define an abstract class with the ...
0
votes
1
answer
146
views
Can a database wrapper be implemented in Scala using Traits? Is there a better way?
I've decided to implement a database wrapper to provide data operations of a web service using postgresql dbms.
Models in my database are somelike complex and frameworks like slick do provide more ...
- 152
5
votes
2
answers
1k
views
Avoiding instanceof for recursive data types
I have written a simple class hierarchy to represent terms in Scala. Terms are recursive data types, e.g., a Sum and a Multiplication consist of the left-hand-side (lhs), which is a Term, and the ...
- 159
3
votes
2
answers
939
views
Reverse engineering an ETL pipeline
I have a one-way large ETL pipeline in Scala where I start with protobuf schema A and end up with protobuf schema B. I want to create a many-to-many mapping between the fields of schema A and schema B,...
- 109
1
vote
1
answer
127
views
Simplified API one case class vs robust and multi ADT case class? [duplicate]
Below are oversimplified examples, but I wonder which route to take what is the best practice when designing API, the simple or the more complex and robust which is better?
This looks good and goes ...
- 507
0
votes
1
answer
891
views
RESTful API with Scala using Jersey?
Thanks Maven archetypes the barrier to start developing of services for new RESTful APIs is very straightforward - you can get more or less a project template and start filling in your code.
Now as I ...
- 185
1
vote
1
answer
428
views
Modelling relational database entities in a functional language
I'm working on a Scala project that uses DynamoDB for persistence, and does this by modelling the records as case classes.
This is becoming increasingly more relational, which means we have classes ...
- 121
1
vote
2
answers
73
views
In my use case, does it make sense to make every major method return an async result?
I have a Playframework application written in Scala. As you might expect, it's primary function is to respond to user requests. Typical request flow:
Parse input and figure out what the user wants ...
1
vote
2
answers
626
views
Why doesn't scala do type checking when checking for equality
For example:
scala> val a: String = "5"
a: String = 5
scala> val b: Int = 5
b: Int = 5
scala> a == b
res5: Boolean = false
I expect a TypeError, like in Java:
class MyClass {
public ...
- 455
3
votes
2
answers
255
views
functional programming: impact of typedef-ing datatypes on code readability and maintenance
In functional programming languages, such as Scala, data types and structures, are really important. I am in two minds about the use of type-defs in helping with the readability of the code ...
- 283
2
votes
2
answers
459
views
Designing Scala library which wraps a Java library
I'm designing a simple scala wrapper library over a java library in order to make it more idiomatic scala. The problem I faced with is that all operations from the java library could throw an ...
- 183
3
votes
1
answer
1k
views
Should I pass all arguments to a method explicitly in functional programming?
I wonder whether a method in a functional programming language should receive all variables from the argument list, or whether it is ok to use variables from the outer scope?
But let me explain the ...
- 139
3
votes
2
answers
907
views
Using actor model, how can one program concurrent portion (critical section) of code as self contained nuggets?
Von neumann architecture allows sequential processing of instructions. So, a single core within a CPU executes instructions sequentially.
Consider, OS providing 1-1 threading model(here) in a multi-...
- 2,315
2
votes
1
answer
165
views
Placing case classes
If the supervisor and its child actors are sharing the same messages, which is the right place to keep those case classes? Should it be in both Supervisor and the individual actors or should it be in ...
- 129
4
votes
1
answer
692
views
Scala Callback Pyramid of Doom
I would like to solicit some general design principles and best practices to avoid creating a callback pyramid of doom particularly in the Scala language.
Consider the following rudimentary and ...
- 727
4
votes
2
answers
3k
views
How to make a webdriver run reliably in Selenium?
I have been having quite a time getting this to work reliably for 100s of thousands of terms and potentially millions of pages per source and ETL the resulting data into a database in an automated ...
- 197
26
votes
6
answers
7k
views
Why use Either over (checked) Exception?
Not too long ago I started using Scala instead of Java. Part of the "conversion" process between the languages for me was learning to use Eithers instead of (checked) Exceptions. I've been coding this ...
- 623