Scala lazy elements in iterator

Question

Does anyone know how to create a lazy iterator in scala?

For example, I want to iterate through instantiating each element. After passing, I want the instance to die / be removed from memory.

If I declare an iterator like so:

val xs = Iterator(
 (0 to 10000).toArray,
 (0 to 10).toArray,
 (0 to 10000000000).toArray)

It creates the arrays when xs is declared. This can be proven like so:

def f(name: String) = {
  val x =  (0 to 10000).toArray
  println("f: " + name) 
  x
}

val xs = Iterator(f("1"),f("2"),f("3"))

which prints:

scala> val xs = Iterator(f("1"),f("2"),f("3"))
f: 1
f: 2
f: 3
xs: Iterator[Array[Int]] = non-empty iterator

Anyone have any ideas?

Streams are not suitable because elements remain in memory.

Note: I am using an Array as an example, but I want it to work with any type.

This hack seems to perform as I want: val it = List(() => g("1"), () => g("2"), () => g("3")).toIterator.map(_()) — S0rin
– S0rin, Commented Jan 18, 2013 at 11:08

Community · Accepted Answer · 2017-05-23 11:46:46Z

2

Scala collections have a view method which produces a lazy equivalent of the collection. So instead of (0 to 10000).toArray, use (0 to 10000).view. This way, there will be no array created in the memory. See also https://stackoverflow.com/a/6996166/90874, https://stackoverflow.com/a/4799832/90874, https://stackoverflow.com/a/4511365/90874 etc.

edited May 23, 2017 at 11:46

CommunityBot

11 silver badge

answered Jan 18, 2013 at 10:48

thSoft

22.8k6 gold badges96 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

S0rin Over a year ago

If I had them in a list, they would remain in memory for the duration of the list. I only want them in memory when I pass over that element.

S0rin Over a year ago

I am using an Array as an example, but I want it to work with any type.

Randall Schulz Over a year ago

@SomeoneElse: Having been placed in a List does not imply being ineligible for reclamation by the garbage collector. If you build a list of 10 items and drop the first 5, e.g., then those 5 cons cells will be garbage, as will the content values (assuming nothing else retains references to those list cells or their contained values, of course).

incrop · Accepted Answer · 2013-01-18 11:15:57Z

1

Use one of Iterator factory methods which accepts call-by-name parameter.

For your first example you can do one of this:

val xs1 = Iterator.fill(3)((0 to 10000).toArray)
val xs2 = Iterator.tabulate(3)(_ => (0 to 10000).toArray)
val xs3 = Iterator.continually((0 to 10000).toArray).take(3)

Arrays won't be allocated until you need them.

In case you need different expressions for each element, you can create separate iterators and concatenate them:

val iter = Iterator.fill(1)(f("1")) ++ 
           Iterator.fill(1)(f("2")) ++ 
           Iterator.fill(1)(f("3"))

answered Jan 18, 2013 at 11:15

incrop

2,7381 gold badge19 silver badges17 bronze badges

Collectives™ on Stack Overflow

Scala lazy elements in iterator

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related