node.js - Module request how to properly retrieve accented characters? ? ? ?

Question

I'm using: Module: Request -- Simplified HTTP request method to scrape a webpage with accented characters á é ó ú ê ? etc.

I've already tried encoding: utf-8 with no success. I'm still getting this ??? characters in the result.

request.get({
    uri: url,
    encoding: 'utf-8'
    // ...

Is there any configuration to fix it?

I don't know if it is an issue, but I filled one for this module. No answers yet. :/

Answer 1

Since binary is deprecated it seems like a better idea to use iconv and correctly handle the decoding:

var request = require("request"), iconv  = require('iconv-lite');
var requestOptions  = { encoding: null, method: "GET", uri: "http://something.com"};

request(requestOptions, function(error, response, body) {
    var utf8String = iconv.decode(new Buffer(body), "ISO-8859-1");
    console.log(utf8String);
});

The important part is to set the encoding on the HTTP request to be null encoding: null.