remove duplicate rows in mysql

ALTER IGNORE TABLE jobs ADD UNIQUE INDEX idx_name (site_id, title, company );\

http://stackoverflow.com/questions/3311903/remove-duplicate-rows-in-mysql

usefull regular expression

/\*(.|[\r\n])*?\*/   for removing comments
^\s*\n  for removing blank lines
//.*$   for single line comment

\d+:  for line removing