1

Verify all links of a site using wget's log

wget -S --recursive -a ./log SOME_URL -l 99
sed -n '/.*=>.*/h;/.*HTTP.*4.*/{H;g;p};' log

example output

           => `192.168.5.16/robots.txt'
 1 HTTP/1.1 404 Not Found
           => `192.168.5.16/cgi-bin/proxyjudge2.35.pl'
 1 HTTP/1.1 404 Not Found

how it works

/.*=>.*/h

Find lines matching the REGEXP and write the result into the buffer.

/.*HTTP.*4.*/

Find lines matching the REGEXP.
Note that SED is not using the buffer as input, it's still using the content of ./log!

{H;g;p}

H: append the output to the buffer
g: overwite "pattern space" with the content of the buffer
p: print the "pattern space"