java StringTokenizer unexpected results

Question

i have the following code which will tokenize the string to create list of Objects:

import java.util.StringTokenizer;


public class TestStringTokenizer {
    private final static String INTERNAL_DELIMETER = "#,#";
    private final static String EXTERNAL_DELIMETER = "#|#";
    public static void main(String[]aregs){
        String test= "1#,#Jon#,#176#|#2#,#Jack#,#200#|#3#,#Jimmy#,#160";
        StringTokenizer tokenizer = new StringTokenizer(test, EXTERNAL_DELIMETER);
        while(tokenizer.hasMoreElements()){
            System.out.println(tokenizer.nextElement());
            //later will take this token and extract elements
        }
    }
}

What i expected output was
1#,#Jon#,#176
2#,#Jack#,#200
3#,#Jimmy#,#160

What i got was : 1
,
Jon
,
176
2
,
Jack
,
200
3
,
Jimmy
,
160

if i change the internal delimeter to some thing like , it will work properly why is this behavior happening ?

StringTokenizer doesn't take #|# as one single delimiter, but as 3 delimiters. Kindly go through the API of that class to understand how it works. — Rohit Jain
– Rohit Jain, Commented Mar 9, 2014 at 7:42

aliteralmind · Accepted Answer · 2014-03-09 13:47:51Z

5

AccordIng to the StringTokenizer JavaDocs

http://docs.oracle.com/javase/7/docs/api/java/util/StringTokenizer.html

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

Use String.split instead:

String[] strArr = stringToSplit.split(INTERNAL_DELIMETER);

The only change you need to make is that the or-pipe ("|") in EXTERNAL_DELIMETER is a special regular expression character, and must be escaped: "\\|".

More information can be found in the String.split Javadoc:

http://docs.oracle.com/javase/7/docs/api/java/lang/String.html#split(java.lang.String)

edited Mar 9, 2014 at 13:47

answered Mar 9, 2014 at 7:48

aliteralmind

20.2k17 gold badges80 silver badges109 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Evgeniy Dorofeev · Accepted Answer · 2014-03-09 11:51:03Z

4

StrinTokenizer can't work with expression as a delimiter, try Scanner instead

    Scanner sc = new Scanner(test);
    sc.useDelimiter("#\\|#");
    while (sc.hasNext()) {
        System.out.println(sc.next());
    }

edited Mar 9, 2014 at 11:51

answered Mar 9, 2014 at 7:47

Evgeniy Dorofeev

137k31 gold badges209 silver badges288 bronze badges

Comments

hmashlah · Accepted Answer · 2014-03-09 08:48:31Z

3

StrinTokenizer constructor's second parameter is delimiters(Each character is a delimiter)

You can use String.split instead

public class TestStringTokenizer {
    private final static String INTERNAL_DELIMETER = "#,#";
    private final static String EXTERNAL_DELIMETER = "#|#";
    public static void main(String[]aregs){
        String test= "1#,#Jon#,#176#|#2#,#Jack#,#200#|#3#,#Jimmy#,#160";
        for (String s : test.split("#\\|#"))
            System.out.println(s);
        }
    }
}

edited Mar 9, 2014 at 8:48

answered Mar 9, 2014 at 7:45

hmashlah

1447 bronze badges

5 Comments

aliteralmind Over a year ago

I am surprised that split only runs once, being in the foreach loop like that. I honestly wouldn't put it there even if it does work. Confusing.

Arjit Over a year ago

its not giving the correct o/p as expected .... I have edited the code...now it works

hmashlah Over a year ago

I added an escape character before | since the split parameter is a regex

hmashlah Over a year ago

The iterator part of the for-each loop is evaluated once to get an iterator

aliteralmind Over a year ago

There are two problems with your code, but they're not really bugs, per se: INTERNAL_DELIMETER is not used at all in your answer, so consider eliminating it. EXTERNAL_DELIMITER is also not being used, but it should be. Instead of calling split with the value of EXTERNAL_DELIMITER, call it with the variable-name (and change the value of EXTERNAL_DELIMITER so its or-pipe is escaped). As it stands, this answer is correct, but written very confusingly. I'll delete this comment once it's fixed. Just respond with @aliteralmind.

Collectives™ on Stack Overflow

java StringTokenizer unexpected results

3 Answers 3

Comments

Comments

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related