Crawl 记忆承载 & 记忆承载3(chinese blogger)'s daily post from http://www.jintiankansha.me. Use airflow to automatically run the crawl-job every day. Also include a script the crawl all historical data.
This should be put under the /dags repo under ~/airflow