NodeJS, CLI, Instagram!

Written by Islom

July 29, 2017

Assalom Alaykum Dunyo! Ba’zida offline paytda chiroyli bino va tabiatli rasmlarni ko‘rgiz kelib qoladi. Yoki instagram akkauntingizni o‘chirmoqchisiz lekin rasmlar kerak bo‘lsa, hamma rasmlarni ko‘chirishga to‘g‘ri keladi. Bugungi postda NodeJSda command line app qilishni o‘rganamiz. Bu cli app Instagramdagi hamma rasmlaringizni ko‘chirib beradigan bo‘ladi.

Yozadigan kodlarimiz githubda bor.

Mundarija

CLI

Process.argv

NodeJSni dokumentatsiyasini o‘qib chiqgan bo‘lsangiz, require() siz ham ishlaydigan process degan global object bor. Bu objectni argv massividan foydalanib cli dasturchalar tuzsa bo‘ladi. Bu massivni birinchi elementi har doim process.execPathga teng. Aniqroq chunish uchun misol keltiramiz:

pargv.js faylni yaratib uni ichiga bu kodni yozsak:

process.argv.forEach((arg, index) => {
  console.log(`${index}: ${arg}`);
});

Keyin NodeJS processni bunday qilib ishga tushirsak:

$ node pargv one two=three four

Process menda mana bunday javob qaytaradi:

0: /usr/local/bin/node
1: /Users/isa/test/node/pargv.js
2: one
3: two=three
4: four

Commander

Agar siz oddiyroq yoki o‘zingiz uchun cli dastur qilmoqchi bo‘lsangiz process.argv bilan qilgan maqul, ishingiz tez bitadi.

Agar boshqalar ham ishlatishni xohlasangiz chunarliroq ko‘rinishda qilish kerak. --help argumentini albatta qilish kerak. Buni process.argv bilan qilish mumkin, lekin vaqt sarflashga to‘g‘ri keladi. Agar vaqt sarflashni xohlamasangiz TJ Holowaychuk yaratgan commander modulidan foydalanish mumkin. Bu modulda ko‘p narsalar avtomatlashgan, qulay va onsonlashtirilgan.

NPM dan commanderni ko‘chirib olamiz, pargv.jsga o‘zgartirishlar kiritamiz va tekshirib ko‘ramiz

$ npm i commander 

var program = require('commander');

program
    .arguments('<file>')
    .option('-u, --username <username>',  'Username yoziladi')
    .option('-p, --password <password>', 'User ni paroli yoziladi')

program.action(function(file) {
        // console.log(file);
        doJob(program.username, program.password, file)
    }
);

program.parse(process.argv);

function doJob(u, p, file) {
    console.log("Username " + u)
    console.log("Password " + p)
    console.log("File     " + file)
}


$ node pargv --help

  Usage: pargv [options] <file>


  Options:

    -u, --username <username>  Username yoziladi
    -p, --password <password>  User ni paroli yoziladi
    -h, --help                 output usage information

Instagram

@hamidullakhodja/media

URLni oxirigia /media qo‘shib oxirgi 20ta postni tayyor JSON formatda olish mumkin: https://instagram.com/hamidullakhodja/media.

Agar hamma rasm kerak bo‘lsachi? Hamma postlar tugamaguncha siklda o‘sha 20ta rasmning oxirgi rasmining idsini URL paramertga max_id=id qilib qo‘shib jo‘natsak bo‘ladi: https://instagram.com/hamidullakhodja/media?max_id=id. Qulaylik uchun JSONda more_available polya ham bor, true va false qiymat oladigan.

Rekursiv metod

Keling endi NodeJSda amalda qo‘llab ko‘ramiz.

Odatda men http requestlar uchun request modulini ishlataman. Berilgan misolda http requestlar shu modul orqali bo‘ladi:

npm i request

Bitta rasmni ko‘chirib olish:

var request = require("request");
var max_id = "";

main();

function main() {
    request({
        url: "http://instagram.com/hamidullakhodja/media?max_id=" + max_id,
        method: "GET",
        rejectUnauthorized: true,
        requestCert: true,
        agent: false,
        json: true
    }, function (error, response, body) {
        if (!error && response.statusCode === 200) {
            console.log(body);
            request(body.items[0].images.standard_resolution.url)
                    .pipe(require('fs').createWriteStream(body.items[0].id + ".jpg"));

        } else {
            console.log(error);
        }
    });
}

Hamma rasmlarni ko‘chirib olishni rekursiv usulini ko‘rib chiqamiz. Funksiya, birinchi JSONni olib kelish uchun GET request yuboradi va max_id o‘zgaruvchisini oxirgi rasmni idsi bilan tenglashtiradi. 20ta rasmni hammasini ko‘chirib oladi va oxirida more_available true yoki false ligini tekshiradi. true bo‘lsa rasm yana borligini bildiradi va shu funksiyani yana ishlatib yuboradi. false bo‘lsa, rekursiyani to‘xtatadi. 

var request = require("request");
var max_id = "";
var account = process.argv[2];

require('fs').mkdirSync(account);

main();
var request_number = 0;

function main() {
    request({
        url: "http://instagram.com/" + account + "/media?max_id=" + max_id,
        method: "GET",
        rejectUnauthorized: true,
        requestCert: true,
        agent: false,
        json: true
    }, function (error, response, body) {
        if (!error && response.statusCode === 200) {
            request_number++;
            max_id = body.items[body.items.length-1].id;
            for(var i = 0; i < body.items.length; i++) {
                console.log('kichkina request');
                request(body.items[i].images.standard_resolution.url).pipe(require('fs').createWriteStream("./"+ account + "/" + body.items[i].id + ".jpg"));
            }

            console.log("tugadi kichkina request");
            if(body.more_available) {
                main(); //more_available true bo'lsa yana funksiyani ishlatamiz
            } else {
                console.log("request numbers", request_number);
                return; //rekursiv funksiyadan chiqamiz
            }
        } else {
            console.log(error);
        }
    });
}


$ node pargv.js hamidullakhodja

Async.js metod

Async degan modul ham bor. Asyncning doUntil funksiyasidan foydalanib ham bajarsa bo‘ladi. Agar kim Pascal(:facepalm:)ni bilsa do Until degan expressionni eslasa kerak 

var request = require("request");
var async = require('async');
var max_id = "";
var account = process.argv[2];
var fs = require('fs');
fs.mkdirSync(account);

available = false;
async.doUntil(
    function(callback) {
        request({
            url: "http://instagram.com/" + account + "/media?max_id=" + max_id,
            method: "GET",
            rejectUnauthorized: true,
            requestCert: true,
            agent: false,
            json: true
        }, function(error, response, body) {
            if(!error && response.statusCode === 200) {
                available = body.more_available
                max_id = body.items[body.items.length-1].id;
                for(var i = 0; i < body.items.length; i++) {
                    request(body.items[i].images.standard_resolution.url)
                    .pipe(require('fs').createWriteStream("./"+ account + "/" + body.items[i].id + ".jpg"));
                }
                    callback(null, available);
            } else {
                callback(error);
            }
        });
    }, 
    function() {
        if(available) {
            return false;
        } else {
            return true;
        }
    }, function(error, status) { 
        if (error) {
            return console.log(error);
        }
    }
);

Avtorizatsiya (Shu yerda muammo bor..)

Hozirgi cli dasturchamizda yopiq profildan rasmlarni ko‘chirolmaydi. Buni uchun avtorizatsiya kerak. Instagram avtorizatsiyasini Fiddler 4 dasturi bilan titkilab https://instagram.com/accounts/login/ajax orqali o‘tishini bilib oldim. Lekin negadir menda o‘xshamayapti. Kimdir o‘xshatgan bo‘lsa yoki xatoimni topsa iltimos izoh qoldiring yoki menga telegram orqali yozishingiz mumkin. 

var request =require('request');
var cookie = require('cookie');

request({
    url: "https://instagram.com",
      method: "GET",
      rejectUnauthorized: true,
    requestCert: true,
    agent: false
}, function(error, response, body) {
    if(!error && response.statusCode === 200) {
        var csrftoken = cookie.parse(response.headers['set-cookie'][2]).csrftoken;
        var mid = cookie.parse(response.headers['set-cookie'][3]).mid;
        console.log("mid", mid);
        console.log("csrftoken", csrftoken);
        // var cookies = j.getCookies('https://instagram.com');
        // console.log("COOK", cookies)

        var header = {
            "cookie" : "csrftoken=" + csrftoken + "; mid="+ mid + ";",
            "Connection": "keep-alive",
            "Origin" : "https://instagram.com",
            "X-Instagram-AJAX": 1,
            "Accept-Language": "en-US,en;q=0.8",
            "Accept": "*/*",
            "Referer" : "https://instagram.com/accounts/login",
            "X-CSRFToken" : csrftoken,
            "User-Agent" : "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36"
        }
        // var j = request.jar();
        // var cookie_mid = request.cookie("mid=" + mid);
        // var cookie_csrftoken = request.cookie("csrftoken=" + csrftoken);

        // j.setCookie(cookie_mid, "https://www.instagram.com/accounts/login/ajax/")
        // j.setCookie(cookie_csrftoken, "https://www.instagram.com/accounts/login/ajax/")
        request({
            url: "https://www.instagram.com/accounts/login/ajax/",
              method: "POST",
              headers : header,
              form : {
                  username : 'hamidullakhodja',
                password : '************************'
              },
              // jar : j,
              rejectUnauthorized: true,
            requestCert: true,
            agent: false
        }, function(error, response, body) {
            if(!error && response.statusCode === 200) {
                console.log(body);
            } else {
                console.log(error);
            }
            console.log(response.statusCode)
            console.log(response.socket._httpMessage._header) // request header
            console.log(response.headers) // kelgan javobni headeri

        });
    } else {
        console.log(error)
    }

});

I have a pen, I have an apple. Ugghh…

NPM INITIALIZE

Endi papka ochib haqiqiy cli app tuzishni boshlaymiz.

$ mkdir instacli
$ cd instacli
$ npm init

Keyin package.json sozlamalarini kiritish kerak:

name: (instacli) [Enter]
version: (1.0.0) 0.0.1 [Enter]
description: Bu yerda app haqida bo'ladi [Enter]
entry point: (index.js) cli.js [Enter]
test command: --help [Enter]
git repository: [Enter]
keywords: [Enter]
author: HamidullaKhodja [Enter]

cli.js faylni yaratamiz va yoqtirgan redaktoringizda ochamiz

$ touch cli.js

Bilimlarni aralashtirish vaqti keldi

Ikki xil funksiya bo‘ladi. Birinchisi ochiq akkauntliklarni rasmlari bo‘ladi. Ikkinchisi yopiq akkaunliklarni rasmlarini ko‘chirishga bo‘ladi. download() va download_auth().

#!/usr/bin/env node

var program = require('commander');
var request = require("request");
var fs = require('fs');
var max_id = "";


program
    .arguments('<account>')
    .option('-u, --username <username>',  'Avtorizatsiya uchun Username yoziladi')
    .option('-p, --password <password>', 'Avtorizatsiya uchun User ni paroli yoziladi')

program.action(function(account) {
    if( typeof program.username === 'undefined' && typeof program.password === 'undefined' ) {
        if(!fs.existsSync(account)) fs.mkdirSync(account);
        download(account);
    } else if( typeof program.username !== 'undefined' && typeof program.password !== 'undefined' ) { //
        download_auth(program.username, program.password, account);
    } else {
        console.log("Iltimos user yoki password'larniyam yozing");

    }

});

program.parse(process.argv);


function download(account, dir) {
    request({
        url: "http://instagram.com/" + account + "/media?max_id=" + max_id,
        method: "GET",
        rejectUnauthorized: true,
        requestCert: true,
        agent: false,
        json: true
    }, function (error, response, body) {
        if (!error && response.statusCode === 200) {            
            if(body.items.length !== 0) {
                max_id = body.items[body.items.length-1].id;
                for(var i = 0; i < body.items.length; i++) {
                    console.log(body.items[i].images.standard_resolution.url);
                    request(body.items[i].images.standard_resolution.url).pipe(fs.createWriteStream("./"+ account + "/" + body.items[i].id + ".jpg"));
                }

                if(body.more_available) {
                    download(account); //more_available true bo'lsa yana funksiyani ishlatamiz
                } else {
                    console.log("Hamma rasmlarni ko'chirib bo'ldi!");
                    return; //rekursiv funksiyadan chiqamiz
                }
            } else {
                console.log("Kechirasiz afsuski bu sahifa yopiq, avtorizatsiya qilib kiring");
                fs.rmdirSync(account)
            }

        } else {
            if (error) console.log(error);
            fs.rmdirSync(account);
        }
    });
}

function download_auth(username, password, account) {
    console.log(username);
    console.log(password)
    console.log(account);
}

Kodimizni birinchi qatorida yozilgan #!/usr/bin/env node yozilgan kod esdan chiqmasligi kerak. Buni shebang deyishadi. CLI dasturchamizni global qilish uchun kerak. (Windows ishlatadiganlar ham yozishi kerak, bo‘lmasam global bo‘lmaydi ;-) )

Kerakli modullarni ko‘chirib birdaniga package.jsonga saqlab qo‘yamiz.

$ npm i request --save
$ npm i commander --save

CLI dasturchamiz deyarli tayyor bo‘ldi!

Globallashtiramiz

Dasturchamizni hozirgi holati judayam noqulay. Faqat bir papkadan ishlatsa bo‘ladi va o‘sha papkaga tushadi rasmlar. Har safar node cli.js qilib ishga tushirishga to‘g‘ri keladi.

Buni to‘g‘irlashning yo‘li bitta, modulni global qilish kerak.

package.jsonga o‘zgartish kiritishga to‘g‘ri keladi. Unga yangi bin va preferGlobal degan polya kiritishga to‘g‘ri keladi:

{
  "name": "instacli",
  "version": "0.0.1",
  "description": "Bu yerda app haqida bo'ladi",
  "main": "cli.js",
  "scripts": {
    "test": "--help"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "commander": "^2.11.0"
  }, 
  "bin" : {
    "instacli": "./cli.js"
  },
  "preferGlobal" : true
}

E’tibor bergan bo‘lsangiz bin ichiga global modul qanday nom bilan chaqirilishi va chaqirilganda qaysi faylni ishga tushirishini ham yozib ketdik.

Global qilish uchun oxirgi qadam:

$ npm i -g

Tekshirib ko‘rish uchun ishlatib ko‘ramiz

$ instacli --help

  Usage: cli [options] <account>


  Options:

    -u, --username <username>  Avtorizatsiya uchun Username yoziladi
    -p, --password <password>  Avtorizatsiya uchun User ni paroli yoziladi
    -h, --help                 output usage information

Ana tayyor bo‘ldi!!

comments powered by Disqus