Create a new list of strings using another but without duplicates

Question

I've looked at multiple different answer and questions but none seem to be giving the result I would like. my issue is that I have a .txt file with 426458 states (disregarding the rules) of noughts and crosses with the format shown below.

[E][E][E]
[E][E][E]
[E][E][E]

[X][X][X]
[O][O][O]
[E][E][E]

I load the file using this:

Scanner sc = new Scanner(new File("src/AllPlayerOneStates.txt"));
    sc.useDelimiter("\r");
    List<String> lines = new ArrayList<String>();
    while (sc.hasNextLine()) {
      lines.add(sc.next());
    }

This creates a list with every element being a state then I would like to create a new list without duplicate states.

I've tried using StringBuilder to simply output all the unique states but it gives me a number much less than expected aswell as looping through each element and comparing it to all the rest but I may have done it wrong as it told me the amount of unique states was larger than 426458. I'm relatively new to programming.

"This creates an array ..." The code you show does not use arrays at all. — Lew Bloch
– Lew Bloch, Commented Mar 1, 2017 at 20:03
They are similar, in some ways. But "array" is a specific thing, and your code doesn't use that specific thing. Words matter, especially words with exact definitions in the JLS. — Lew Bloch
– Lew Bloch, Commented Mar 1, 2017 at 20:14

VHS · Accepted Answer · 2017-03-01 19:45:08Z

3

This creates an array with every element being a state then I would like to create a new array without duplicate states.

Try the following way:

Set<String> set = new HashSet<String>(lines);
List<String> newList = new ArrayList<String>(set);

Here newList is your new List of states that doesn't contain duplicates.

answered Mar 1, 2017 at 19:45

VHS

10.2k3 gold badges25 silver badges48 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Darshan Mehta · Accepted Answer · 2017-03-01 19:42:28Z

2

You can use Set instead of List and it will remove the duplicates itself, e.g.:

Set<String> lines = new HashSet<String>();
while (sc.hasNextLine()) {
  lines.add(sc.next());
}

Once this while loop finishes, lines.size() will give you number of unique rules.

If you want to preserve the order, you can use LinkedHashSet instead of HashSet.

answered Mar 1, 2017 at 19:42

Darshan Mehta

31k14 gold badges75 silver badges111 bronze badges

4 Comments

Alistair Osborne Over a year ago

it must be the way im formatting my text file as it gives me the same result of only 28 unique states when there should be much more

Darshan Mehta Over a year ago

try printing out each element as it gets added into the set, that might give you more idea on what's being added..

Lew Bloch Over a year ago

You will only get the right result if equals and hashCode correctly compare states. Does one line of input really correspond to a single state?

Alistair Osborne Over a year ago

this is what the hash list looks like after being read into a file pastebin.com/4rEACmqS im not entirely sure what is going wrong or why the 3rd or 4th element is not in the same format

Collectives™ on Stack Overflow

Create a new list of strings using another but without duplicates

2 Answers 2

Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related