Re: [問題] 郵局網路爬蟲輸出csv問題

[LP9527 ()]

我也睡完午覺了 真不知道為啥廢文不自Ｄ

以下給原ＰＯ參考

---

import bs4
import urllib.request as req
import pandas as pd
from pathlib import Path


url = "https://www.post.gov.tw/post/internet/I_location/index_all.html"
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
}


temp_file = 'tmp.html'
if not Path(temp_file).is_file():
    request = req.Request(url, headers=headers)
    with req.urlopen(request) as response:
        data = response.read().decode("utf-8")
    with open(temp_file, mode='w') as f:
        f.write(data)
else:
    with open(temp_file) as f:
        data = f.read()


root = bs4.BeautifulSoup(data, "html.parser")
rows = root.find(id="table").find_all('tr')[1:]  # 表頭不取


output = []
for row in rows:
    addr = row.find('td', class_="detail").string
    row_output = {
        '地址': addr,
        '縣市': addr[:3],
        '局名': row.find('a', class_="rwd-close").string,
    }
    output.append(row_output)

# 請你看看此時 output 長啥樣子
# 最好是熟悉 python 的 list, dict 這兩個資料型態的基本操作之後再去用 df


output_df = pd.DataFrame(output)

csv_file = "test.csv"
output_df.to_csv(csv_file, encoding="utf-8-sig")



※ 引述《Leo33012 (羊圈裡的狼)》之銘言：
: 我等等試完來告訴你，
: 但是要先睡個午覺，
: 你可以等一等
: 你可以留意一下這篇帖子等等的留言

--
※ 發信站: 批踢踢實業坊(ptt.cc), 來自: 223.140.234.255 (臺灣)
※ 文章網址: https://www.ptt.cc/bbs/Python/M.1627798606.A.2F3.html




# 如果你要整個表格直接轉換可以這樣做

root = bs4.BeautifulSoup(data, "html.parser")
table = root.find(id="table").prettify()
output_df = pd.read_html(table, flavor='html5lib')[0]  # pip install


csv_file = "test.csv"
output_df.to_csv(csv_file, encoding="utf-8-sig")

※ 編輯: LP9527 (223.140.234.255 臺灣), 08/01/2021 14:31:38
→ jerrycurry: 感謝感謝
推 jerrycurry: 補個推