Hi,站長大大&眾先進:
關於站長上回所發急策,擋掉某網域之事,小妺認為看來似乎嚴重,於是和舍弟研討了一下.而舍弟認為站長要行如此決定很不得已,因為擋網域會連帶將其它不相關使用者給擋去,使得這些人永遠不得上本站.故而舍弟給了您建議如下:

流量管制!
(若本站的server有支援的話?)
例如可以管每個session只能最多以某Kbps的速度來傳資料,這樣本站的負荷就不會太大.

小妹本學非資訊,只是覺得這個問題影響甚巨,小妮子在此雞婆一下,請別見笑^_^.

From:Coka.

關於站長上回所發急策,擋掉某網域之事,小妺認為看來似乎嚴重,於


沒錯，是相當嚴重，因為這種抓取網站的軟體會在短時間內送出上千個 request (要求)
對於本站爛爛的電腦來說，實在無法負荷
昨天又有人從 Hinet 以 Teleport 抓取,
還好是用 modem 速度不快而且只抓了一會兒, 我也睜隻眼閉隻眼
我可不想把整個 Hinet 都擋掉

我是覺得，您若想拷貝, 您用 copy/paste 也可以,
但若是用這種抓網站的軟體來抓, 造成本站當站, 就太過份了



流量管制!
(若本站的server有支援的話?)
例如可以管每個session只能最多以某Kbps的速度來傳資料,這樣本站的負荷就不會太大.


流量管制? 不錯的idea
但是 , Apache 有支援嗎？
(本站是用 apache 當 server 的)
小弟不才, 好像沒看過類似的設定哦...

我去查查看

Flow-control might not be an ideal solution, since the ultimate problem is not
the server being incapable of supporting the network traffic, but rather,
extensive use of database engine exhausts system resources.

Flow-control is usually used by ISP (supporting virtual domain hosting) on per IP
address basis, so theoretically it shouldn't be part of webserver software. And
the session-based flow-control as mentioned (I don't know if such thing ever exists)
is very difficult to implement, since http is basically a stateless protocol, each
URL request is a unique session to the webserver.

A would-be solution might be reducing the max process limit of httpd
(edit httpd.conf) to a suitable value such that even if all httpd processes
are busy querying the database, the system will still be functioning.

Of course, this will only safeguard the proper functioning of the server,
all subsequent requests from other users might have difficulties connecting
to server.

Flow-control might not be an ideal solution, since the ultimate problem is not
the server being incapable of supporting the network traffic, but rather,
extensive use of database engine exhausts system resources.


wptan 說得沒錯, 以 Teleport 抓取本站內容，影響最鉅的，是急速消耗後端資料庫的資源
如果本站的內容大多都是靜態的資料，那相信不管大家怎麼抓取，都不太會有問題
但重點是本站的資料 99% 都是各位討論的內容，存於本超級討論區，是動態產生的
在本站硬體設備不夠好的前提下，是禁不起 Teleport 的「摧殘」的！


A would-be solution might be reducing the max process limit of httpd
(edit httpd.conf) to a suitable value such that even if all httpd processes
are busy querying the database, the system will still be functioning.

Of course, this will only safeguard the proper functioning of the server,
all subsequent requests from other users might have difficulties connecting
to server.


唉，只要不有人用 Teleport 抓取，一切都沒事...也不用設什麼限制
但就是有人這麼講不聽...

Hi,站長大大:
很抱歉小妺現在才回覆,近來較忙.*-*

關於上回您的回答,小妺又和舍弟討論過了,舍弟願意和您談談解決方案,我會直接寫他的E-mail寄到您的信箱裡,您再和我弟弟討論看看如何?

現在舍弟也是資管研究生,我想站長大大也是的樣子,或許你們可以直接talk talk看看比較明白.

From:Coka.