remove duplicate lines with regex in javascript

Question

According to regular-expressions.info, I should be able to remove duplicate lines (as long as they are consecutive) with the following pattern: ^(.*)(\r?\n\1)+$

This is pretty straightforward within Notepad++, I just hit "Replace all" with \1 and all the duplicates are removed. However, I cannot get it to work with javascript when pulling text from a <textarea>.

var t = $("#input").val();
var re = /^(.*)(\r?\n\1)+$/g;
var s = t.replace(re, "$1");
console.log(s);

Why has adding the g flag not removed all of the duplicate lines?

You can try this regex:- ^(.*)(?:(?:\r?\n|\r)\1)+$

Rahul Tripathi
– Rahul Tripathi

2013-10-01 15:01:11 +00:00
Commented Oct 1, 2013 at 15:01 — Rahul Tripathi
– Rahul Tripathi, Commented Oct 1, 2013 at 15:01

Jerry · Accepted Answer · 2013-10-01 15:11:01Z

2

Use the multiline modifier as well:

var re = /^(.*)(\r?\n\1)+$/gm;

It seems that your input contains several lines and you want to remove some lines in the whole text which are duplicates.

One condition in that regex is that the definition of a 'line' is actually a string in its own line (there's no other string occupying any part of that line).

As such, the multiline modifier (m) will make the ^ and $ match the beginning and end of each individual line as opposed to the whole text.

answered Oct 1, 2013 at 15:11

Jerry

71.8k14 gold badges106 silver badges148 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

sp00m Over a year ago

It seems to work without the m flag actually (fiddle), so not sure this is the problem.

Jerry Over a year ago

@sp00m Nope, it's not working for me, see: the first link. But when I add the multiline more, I get second link

Niet the Dark Absol · Accepted Answer · 2013-10-01 15:00:33Z

0

The correct replace call is:

var s = t.replace(re, "$1");

The replacement must be a string (or a function, for more complex stuff)

answered Oct 1, 2013 at 15:00

Niet the Dark Absol

326k86 gold badges480 silver badges604 bronze badges

1 Comment

1252748 Over a year ago

Ah, that was a typo that it wasn't in quotes.

Collectives™ on Stack Overflow

remove duplicate lines with regex in javascript

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related