gopl-zh.github.com/ch8/ch8-06.html

2208 lines
78 KiB
HTML
Raw Normal View History

2015-12-09 07:57:17 +00:00
<!DOCTYPE HTML>
<html lang="zh-tw" >
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
2015-12-21 04:55:18 +00:00
<title>示例: 併發的Web爬蟲 | Go编程语言</title>
2015-12-09 07:57:17 +00:00
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<meta name="description" content="">
<meta name="generator" content="GitBook 2.5.2">
<meta name="HandheldFriendly" content="true"/>
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black">
<link rel="apple-touch-icon-precomposed" sizes="152x152" href="../gitbook/images/apple-touch-icon-precomposed-152.png">
<link rel="shortcut icon" href="../gitbook/images/favicon.ico" type="image/x-icon">
<link rel="stylesheet" href="../gitbook/style.css">
<link rel="stylesheet" href="../gitbook/plugins/gitbook-plugin-highlight/website.css">
<link rel="stylesheet" href="../gitbook/plugins/gitbook-plugin-search/search.css">
<link rel="stylesheet" href="../gitbook/plugins/gitbook-plugin-fontsettings/website.css">
<link rel="next" href="../ch8/ch8-07.html" />
<link rel="prev" href="../ch8/ch8-05.html" />
</head>
<body>
2015-12-24 06:47:06 +00:00
<div class="book" data-level="8.6" data-chapter-title="示例: 併發的Web爬蟲" data-filepath="ch8/ch8-06.md" data-basepath=".." data-revision="Thu Dec 24 2015 14:42:02 GMT+0800 (中国标准时间)">
2015-12-09 07:57:17 +00:00
<div class="book-summary">
<nav role="navigation">
<ul class="summary">
<li class="chapter " data-level="0" data-path="index.html">
<a href="../index.html">
<i class="fa fa-check"></i>
前言
</a>
<ul class="articles">
<li class="chapter " data-level="0.1" data-path="ch0/ch0-01.html">
<a href="../ch0/ch0-01.html">
<i class="fa fa-check"></i>
<b>0.1.</b>
Go語言起源
</a>
</li>
<li class="chapter " data-level="0.2" data-path="ch0/ch0-02.html">
<a href="../ch0/ch0-02.html">
<i class="fa fa-check"></i>
<b>0.2.</b>
Go語言項目
</a>
</li>
<li class="chapter " data-level="0.3" data-path="ch0/ch0-03.html">
<a href="../ch0/ch0-03.html">
<i class="fa fa-check"></i>
<b>0.3.</b>
本書的組織
</a>
</li>
<li class="chapter " data-level="0.4" data-path="ch0/ch0-04.html">
<a href="../ch0/ch0-04.html">
<i class="fa fa-check"></i>
<b>0.4.</b>
更多的信息
</a>
</li>
<li class="chapter " data-level="0.5" data-path="ch0/ch0-05.html">
<a href="../ch0/ch0-05.html">
<i class="fa fa-check"></i>
<b>0.5.</b>
2015-12-21 04:55:18 +00:00
致謝
2015-12-09 07:57:17 +00:00
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="1" data-path="ch1/ch1.html">
<a href="../ch1/ch1.html">
<i class="fa fa-check"></i>
<b>1.</b>
入門
</a>
<ul class="articles">
<li class="chapter " data-level="1.1" data-path="ch1/ch1-01.html">
<a href="../ch1/ch1-01.html">
<i class="fa fa-check"></i>
<b>1.1.</b>
Hello, World
</a>
</li>
<li class="chapter " data-level="1.2" data-path="ch1/ch1-02.html">
<a href="../ch1/ch1-02.html">
<i class="fa fa-check"></i>
<b>1.2.</b>
命令行參數
</a>
</li>
<li class="chapter " data-level="1.3" data-path="ch1/ch1-03.html">
<a href="../ch1/ch1-03.html">
<i class="fa fa-check"></i>
<b>1.3.</b>
2015-12-21 04:55:18 +00:00
査找重複的行
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="1.4" data-path="ch1/ch1-04.html">
<a href="../ch1/ch1-04.html">
<i class="fa fa-check"></i>
<b>1.4.</b>
2015-12-21 04:55:18 +00:00
GIF動畵
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="1.5" data-path="ch1/ch1-05.html">
<a href="../ch1/ch1-05.html">
<i class="fa fa-check"></i>
<b>1.5.</b>
穫取URL
</a>
</li>
<li class="chapter " data-level="1.6" data-path="ch1/ch1-06.html">
<a href="../ch1/ch1-06.html">
<i class="fa fa-check"></i>
<b>1.6.</b>
2015-12-21 04:55:18 +00:00
併發穫取多個URL
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="1.7" data-path="ch1/ch1-07.html">
<a href="../ch1/ch1-07.html">
<i class="fa fa-check"></i>
<b>1.7.</b>
Web服務
</a>
</li>
<li class="chapter " data-level="1.8" data-path="ch1/ch1-08.html">
<a href="../ch1/ch1-08.html">
<i class="fa fa-check"></i>
<b>1.8.</b>
本章要點
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="2" data-path="ch2/ch2.html">
<a href="../ch2/ch2.html">
<i class="fa fa-check"></i>
<b>2.</b>
程序結構
</a>
<ul class="articles">
<li class="chapter " data-level="2.1" data-path="ch2/ch2-01.html">
<a href="../ch2/ch2-01.html">
<i class="fa fa-check"></i>
<b>2.1.</b>
命名
</a>
</li>
<li class="chapter " data-level="2.2" data-path="ch2/ch2-02.html">
<a href="../ch2/ch2-02.html">
<i class="fa fa-check"></i>
<b>2.2.</b>
聲明
</a>
</li>
<li class="chapter " data-level="2.3" data-path="ch2/ch2-03.html">
<a href="../ch2/ch2-03.html">
<i class="fa fa-check"></i>
<b>2.3.</b>
變量
</a>
</li>
<li class="chapter " data-level="2.4" data-path="ch2/ch2-04.html">
<a href="../ch2/ch2-04.html">
<i class="fa fa-check"></i>
<b>2.4.</b>
賦值
</a>
</li>
<li class="chapter " data-level="2.5" data-path="ch2/ch2-05.html">
<a href="../ch2/ch2-05.html">
<i class="fa fa-check"></i>
<b>2.5.</b>
類型
</a>
</li>
<li class="chapter " data-level="2.6" data-path="ch2/ch2-06.html">
<a href="../ch2/ch2-06.html">
<i class="fa fa-check"></i>
<b>2.6.</b>
包和文件
</a>
</li>
<li class="chapter " data-level="2.7" data-path="ch2/ch2-07.html">
<a href="../ch2/ch2-07.html">
<i class="fa fa-check"></i>
<b>2.7.</b>
作用域
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="3" data-path="ch3/ch3.html">
<a href="../ch3/ch3.html">
<i class="fa fa-check"></i>
<b>3.</b>
基礎數據類型
</a>
<ul class="articles">
<li class="chapter " data-level="3.1" data-path="ch3/ch3-01.html">
<a href="../ch3/ch3-01.html">
<i class="fa fa-check"></i>
<b>3.1.</b>
整型
</a>
</li>
<li class="chapter " data-level="3.2" data-path="ch3/ch3-02.html">
<a href="../ch3/ch3-02.html">
<i class="fa fa-check"></i>
<b>3.2.</b>
浮點數
</a>
</li>
<li class="chapter " data-level="3.3" data-path="ch3/ch3-03.html">
<a href="../ch3/ch3-03.html">
<i class="fa fa-check"></i>
<b>3.3.</b>
2015-12-21 04:55:18 +00:00
複數
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="3.4" data-path="ch3/ch3-04.html">
<a href="../ch3/ch3-04.html">
<i class="fa fa-check"></i>
<b>3.4.</b>
2015-12-21 04:55:18 +00:00
布爾型
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="3.5" data-path="ch3/ch3-05.html">
<a href="../ch3/ch3-05.html">
<i class="fa fa-check"></i>
<b>3.5.</b>
字符串
</a>
</li>
<li class="chapter " data-level="3.6" data-path="ch3/ch3-06.html">
<a href="../ch3/ch3-06.html">
<i class="fa fa-check"></i>
<b>3.6.</b>
常量
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="4" data-path="ch4/ch4.html">
<a href="../ch4/ch4.html">
<i class="fa fa-check"></i>
<b>4.</b>
2015-12-21 04:55:18 +00:00
複合數據類型
2015-12-09 07:57:17 +00:00
</a>
<ul class="articles">
<li class="chapter " data-level="4.1" data-path="ch4/ch4-01.html">
<a href="../ch4/ch4-01.html">
<i class="fa fa-check"></i>
<b>4.1.</b>
數組
</a>
</li>
<li class="chapter " data-level="4.2" data-path="ch4/ch4-02.html">
<a href="../ch4/ch4-02.html">
<i class="fa fa-check"></i>
<b>4.2.</b>
切片
</a>
</li>
<li class="chapter " data-level="4.3" data-path="ch4/ch4-03.html">
<a href="../ch4/ch4-03.html">
<i class="fa fa-check"></i>
<b>4.3.</b>
字典
</a>
</li>
<li class="chapter " data-level="4.4" data-path="ch4/ch4-04.html">
<a href="../ch4/ch4-04.html">
<i class="fa fa-check"></i>
<b>4.4.</b>
結構體
</a>
</li>
<li class="chapter " data-level="4.5" data-path="ch4/ch4-05.html">
<a href="../ch4/ch4-05.html">
<i class="fa fa-check"></i>
<b>4.5.</b>
JSON
</a>
</li>
<li class="chapter " data-level="4.6" data-path="ch4/ch4-06.html">
<a href="../ch4/ch4-06.html">
<i class="fa fa-check"></i>
<b>4.6.</b>
文本和HTML模闆
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="5" data-path="ch5/ch5.html">
<a href="../ch5/ch5.html">
<i class="fa fa-check"></i>
<b>5.</b>
函數
</a>
<ul class="articles">
<li class="chapter " data-level="5.1" data-path="ch5/ch5-01.html">
<a href="../ch5/ch5-01.html">
<i class="fa fa-check"></i>
<b>5.1.</b>
函數聲明
</a>
</li>
<li class="chapter " data-level="5.2" data-path="ch5/ch5-02.html">
<a href="../ch5/ch5-02.html">
<i class="fa fa-check"></i>
<b>5.2.</b>
遞歸
</a>
</li>
<li class="chapter " data-level="5.3" data-path="ch5/ch5-03.html">
<a href="../ch5/ch5-03.html">
<i class="fa fa-check"></i>
<b>5.3.</b>
多返迴值
</a>
</li>
<li class="chapter " data-level="5.4" data-path="ch5/ch5-04.html">
<a href="../ch5/ch5-04.html">
<i class="fa fa-check"></i>
<b>5.4.</b>
錯誤
</a>
</li>
<li class="chapter " data-level="5.5" data-path="ch5/ch5-05.html">
<a href="../ch5/ch5-05.html">
<i class="fa fa-check"></i>
<b>5.5.</b>
函數值
</a>
</li>
<li class="chapter " data-level="5.6" data-path="ch5/ch5-06.html">
<a href="../ch5/ch5-06.html">
<i class="fa fa-check"></i>
<b>5.6.</b>
匿名函數
</a>
</li>
<li class="chapter " data-level="5.7" data-path="ch5/ch5-07.html">
<a href="../ch5/ch5-07.html">
<i class="fa fa-check"></i>
<b>5.7.</b>
可變參數
</a>
</li>
<li class="chapter " data-level="5.8" data-path="ch5/ch5-08.html">
<a href="../ch5/ch5-08.html">
<i class="fa fa-check"></i>
<b>5.8.</b>
Deferred函數
</a>
</li>
<li class="chapter " data-level="5.9" data-path="ch5/ch5-09.html">
<a href="../ch5/ch5-09.html">
<i class="fa fa-check"></i>
<b>5.9.</b>
Panic異常
</a>
</li>
<li class="chapter " data-level="5.10" data-path="ch5/ch5-10.html">
<a href="../ch5/ch5-10.html">
<i class="fa fa-check"></i>
<b>5.10.</b>
Recover捕穫異常
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="6" data-path="ch6/ch6.html">
<a href="../ch6/ch6.html">
<i class="fa fa-check"></i>
<b>6.</b>
方法
</a>
<ul class="articles">
<li class="chapter " data-level="6.1" data-path="ch6/ch6-01.html">
<a href="../ch6/ch6-01.html">
<i class="fa fa-check"></i>
<b>6.1.</b>
方法聲明
</a>
</li>
<li class="chapter " data-level="6.2" data-path="ch6/ch6-02.html">
<a href="../ch6/ch6-02.html">
<i class="fa fa-check"></i>
<b>6.2.</b>
2015-12-21 04:55:18 +00:00
基於指針對象的方法
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="6.3" data-path="ch6/ch6-03.html">
<a href="../ch6/ch6-03.html">
<i class="fa fa-check"></i>
<b>6.3.</b>
通過嵌入結構體來擴展類型
</a>
</li>
<li class="chapter " data-level="6.4" data-path="ch6/ch6-04.html">
<a href="../ch6/ch6-04.html">
<i class="fa fa-check"></i>
<b>6.4.</b>
2015-12-21 04:55:18 +00:00
方法值和方法表達式
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="6.5" data-path="ch6/ch6-05.html">
<a href="../ch6/ch6-05.html">
<i class="fa fa-check"></i>
<b>6.5.</b>
示例: Bit數組
</a>
</li>
<li class="chapter " data-level="6.6" data-path="ch6/ch6-06.html">
<a href="../ch6/ch6-06.html">
<i class="fa fa-check"></i>
<b>6.6.</b>
封裝
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="7" data-path="ch7/ch7.html">
<a href="../ch7/ch7.html">
<i class="fa fa-check"></i>
<b>7.</b>
接口
</a>
<ul class="articles">
<li class="chapter " data-level="7.1" data-path="ch7/ch7-01.html">
<a href="../ch7/ch7-01.html">
<i class="fa fa-check"></i>
<b>7.1.</b>
2015-12-21 04:55:18 +00:00
接口是合約
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="7.2" data-path="ch7/ch7-02.html">
<a href="../ch7/ch7-02.html">
<i class="fa fa-check"></i>
<b>7.2.</b>
接口類型
</a>
</li>
<li class="chapter " data-level="7.3" data-path="ch7/ch7-03.html">
<a href="../ch7/ch7-03.html">
<i class="fa fa-check"></i>
<b>7.3.</b>
實現接口的條件
</a>
</li>
<li class="chapter " data-level="7.4" data-path="ch7/ch7-04.html">
<a href="../ch7/ch7-04.html">
<i class="fa fa-check"></i>
<b>7.4.</b>
flag.Value接口
</a>
</li>
<li class="chapter " data-level="7.5" data-path="ch7/ch7-05.html">
<a href="../ch7/ch7-05.html">
<i class="fa fa-check"></i>
<b>7.5.</b>
接口值
</a>
</li>
<li class="chapter " data-level="7.6" data-path="ch7/ch7-06.html">
<a href="../ch7/ch7-06.html">
<i class="fa fa-check"></i>
<b>7.6.</b>
sort.Interface接口
</a>
</li>
<li class="chapter " data-level="7.7" data-path="ch7/ch7-07.html">
<a href="../ch7/ch7-07.html">
<i class="fa fa-check"></i>
<b>7.7.</b>
http.Handler接口
</a>
</li>
<li class="chapter " data-level="7.8" data-path="ch7/ch7-08.html">
<a href="../ch7/ch7-08.html">
<i class="fa fa-check"></i>
<b>7.8.</b>
error接口
</a>
</li>
<li class="chapter " data-level="7.9" data-path="ch7/ch7-09.html">
<a href="../ch7/ch7-09.html">
<i class="fa fa-check"></i>
<b>7.9.</b>
2015-12-21 04:55:18 +00:00
示例: 表達式求值
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="7.10" data-path="ch7/ch7-10.html">
<a href="../ch7/ch7-10.html">
<i class="fa fa-check"></i>
<b>7.10.</b>
類型斷言
</a>
</li>
<li class="chapter " data-level="7.11" data-path="ch7/ch7-11.html">
<a href="../ch7/ch7-11.html">
<i class="fa fa-check"></i>
<b>7.11.</b>
2015-12-21 04:55:18 +00:00
基於類型斷言識别錯誤類型
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="7.12" data-path="ch7/ch7-12.html">
<a href="../ch7/ch7-12.html">
<i class="fa fa-check"></i>
<b>7.12.</b>
通過類型斷言査詢接口
</a>
</li>
<li class="chapter " data-level="7.13" data-path="ch7/ch7-13.html">
<a href="../ch7/ch7-13.html">
<i class="fa fa-check"></i>
<b>7.13.</b>
類型分支
</a>
</li>
<li class="chapter " data-level="7.14" data-path="ch7/ch7-14.html">
<a href="../ch7/ch7-14.html">
<i class="fa fa-check"></i>
<b>7.14.</b>
示例: 基於標記的XML解碼
</a>
</li>
<li class="chapter " data-level="7.15" data-path="ch7/ch7-15.html">
<a href="../ch7/ch7-15.html">
<i class="fa fa-check"></i>
<b>7.15.</b>
補充幾點
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="8" data-path="ch8/ch8.html">
<a href="../ch8/ch8.html">
<i class="fa fa-check"></i>
<b>8.</b>
Goroutines和Channels
</a>
<ul class="articles">
<li class="chapter " data-level="8.1" data-path="ch8/ch8-01.html">
<a href="../ch8/ch8-01.html">
<i class="fa fa-check"></i>
<b>8.1.</b>
Goroutines
</a>
</li>
<li class="chapter " data-level="8.2" data-path="ch8/ch8-02.html">
<a href="../ch8/ch8-02.html">
<i class="fa fa-check"></i>
<b>8.2.</b>
2015-12-21 04:55:18 +00:00
示例: 併發的Clock服務
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="8.3" data-path="ch8/ch8-03.html">
<a href="../ch8/ch8-03.html">
<i class="fa fa-check"></i>
<b>8.3.</b>
2015-12-21 04:55:18 +00:00
示例: 併發的Echo服務
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="8.4" data-path="ch8/ch8-04.html">
<a href="../ch8/ch8-04.html">
<i class="fa fa-check"></i>
<b>8.4.</b>
Channels
</a>
</li>
<li class="chapter " data-level="8.5" data-path="ch8/ch8-05.html">
<a href="../ch8/ch8-05.html">
<i class="fa fa-check"></i>
<b>8.5.</b>
併行的循環
</a>
</li>
<li class="chapter active" data-level="8.6" data-path="ch8/ch8-06.html">
<a href="../ch8/ch8-06.html">
<i class="fa fa-check"></i>
<b>8.6.</b>
2015-12-21 04:55:18 +00:00
示例: 併發的Web爬蟲
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="8.7" data-path="ch8/ch8-07.html">
<a href="../ch8/ch8-07.html">
<i class="fa fa-check"></i>
<b>8.7.</b>
2015-12-21 04:55:18 +00:00
基於select的多路複用
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="8.8" data-path="ch8/ch8-08.html">
<a href="../ch8/ch8-08.html">
<i class="fa fa-check"></i>
<b>8.8.</b>
2015-12-21 04:55:18 +00:00
示例: 併發的字典遍歷
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="8.9" data-path="ch8/ch8-09.html">
<a href="../ch8/ch8-09.html">
<i class="fa fa-check"></i>
<b>8.9.</b>
2015-12-21 04:55:18 +00:00
併發的退齣
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="8.10" data-path="ch8/ch8-10.html">
<a href="../ch8/ch8-10.html">
<i class="fa fa-check"></i>
<b>8.10.</b>
示例: 聊天服務
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="9" data-path="ch9/ch9.html">
<a href="../ch9/ch9.html">
<i class="fa fa-check"></i>
<b>9.</b>
2015-12-21 04:55:18 +00:00
基於共享變量的併發
2015-12-09 07:57:17 +00:00
</a>
<ul class="articles">
<li class="chapter " data-level="9.1" data-path="ch9/ch9-01.html">
<a href="../ch9/ch9-01.html">
<i class="fa fa-check"></i>
<b>9.1.</b>
競爭條件
</a>
</li>
<li class="chapter " data-level="9.2" data-path="ch9/ch9-02.html">
<a href="../ch9/ch9-02.html">
<i class="fa fa-check"></i>
<b>9.2.</b>
sync.Mutex互斥鎖
</a>
</li>
<li class="chapter " data-level="9.3" data-path="ch9/ch9-03.html">
<a href="../ch9/ch9-03.html">
<i class="fa fa-check"></i>
<b>9.3.</b>
sync.RWMutex讀寫鎖
</a>
</li>
<li class="chapter " data-level="9.4" data-path="ch9/ch9-04.html">
<a href="../ch9/ch9-04.html">
<i class="fa fa-check"></i>
<b>9.4.</b>
2015-12-21 04:55:18 +00:00
內存同步
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="9.5" data-path="ch9/ch9-05.html">
<a href="../ch9/ch9-05.html">
<i class="fa fa-check"></i>
<b>9.5.</b>
sync.Once初始化
</a>
</li>
<li class="chapter " data-level="9.6" data-path="ch9/ch9-06.html">
<a href="../ch9/ch9-06.html">
<i class="fa fa-check"></i>
<b>9.6.</b>
競爭條件檢測
</a>
</li>
<li class="chapter " data-level="9.7" data-path="ch9/ch9-07.html">
<a href="../ch9/ch9-07.html">
<i class="fa fa-check"></i>
<b>9.7.</b>
2015-12-21 04:55:18 +00:00
示例: 併發的非阻塞緩存
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="9.8" data-path="ch9/ch9-08.html">
<a href="../ch9/ch9-08.html">
<i class="fa fa-check"></i>
<b>9.8.</b>
2015-12-21 04:55:18 +00:00
Goroutines和線程
2015-12-09 07:57:17 +00:00
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="10" data-path="ch10/ch10.html">
<a href="../ch10/ch10.html">
<i class="fa fa-check"></i>
<b>10.</b>
包和工具
</a>
<ul class="articles">
<li class="chapter " data-level="10.1" data-path="ch10/ch10-01.html">
<a href="../ch10/ch10-01.html">
<i class="fa fa-check"></i>
<b>10.1.</b>
簡介
</a>
</li>
<li class="chapter " data-level="10.2" data-path="ch10/ch10-02.html">
<a href="../ch10/ch10-02.html">
<i class="fa fa-check"></i>
<b>10.2.</b>
導入路徑
</a>
</li>
<li class="chapter " data-level="10.3" data-path="ch10/ch10-03.html">
<a href="../ch10/ch10-03.html">
<i class="fa fa-check"></i>
<b>10.3.</b>
包聲明
</a>
</li>
<li class="chapter " data-level="10.4" data-path="ch10/ch10-04.html">
<a href="../ch10/ch10-04.html">
<i class="fa fa-check"></i>
<b>10.4.</b>
導入聲明
</a>
</li>
<li class="chapter " data-level="10.5" data-path="ch10/ch10-05.html">
<a href="../ch10/ch10-05.html">
<i class="fa fa-check"></i>
<b>10.5.</b>
匿名導入
</a>
</li>
<li class="chapter " data-level="10.6" data-path="ch10/ch10-06.html">
<a href="../ch10/ch10-06.html">
<i class="fa fa-check"></i>
<b>10.6.</b>
包和命名
</a>
</li>
<li class="chapter " data-level="10.7" data-path="ch10/ch10-07.html">
<a href="../ch10/ch10-07.html">
<i class="fa fa-check"></i>
<b>10.7.</b>
工具
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="11" data-path="ch11/ch11.html">
<a href="../ch11/ch11.html">
<i class="fa fa-check"></i>
<b>11.</b>
測試
</a>
<ul class="articles">
<li class="chapter " data-level="11.1" data-path="ch11/ch11-01.html">
<a href="../ch11/ch11-01.html">
<i class="fa fa-check"></i>
<b>11.1.</b>
go test
</a>
</li>
<li class="chapter " data-level="11.2" data-path="ch11/ch11-02.html">
<a href="../ch11/ch11-02.html">
<i class="fa fa-check"></i>
<b>11.2.</b>
測試函數
</a>
</li>
<li class="chapter " data-level="11.3" data-path="ch11/ch11-03.html">
<a href="../ch11/ch11-03.html">
<i class="fa fa-check"></i>
<b>11.3.</b>
測試覆蓋率
</a>
</li>
<li class="chapter " data-level="11.4" data-path="ch11/ch11-04.html">
<a href="../ch11/ch11-04.html">
<i class="fa fa-check"></i>
<b>11.4.</b>
基準測試
</a>
</li>
<li class="chapter " data-level="11.5" data-path="ch11/ch11-05.html">
<a href="../ch11/ch11-05.html">
<i class="fa fa-check"></i>
<b>11.5.</b>
剖析
</a>
</li>
<li class="chapter " data-level="11.6" data-path="ch11/ch11-06.html">
<a href="../ch11/ch11-06.html">
<i class="fa fa-check"></i>
<b>11.6.</b>
示例函數
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="12" data-path="ch12/ch12.html">
<a href="../ch12/ch12.html">
<i class="fa fa-check"></i>
<b>12.</b>
反射
</a>
<ul class="articles">
<li class="chapter " data-level="12.1" data-path="ch12/ch12-01.html">
<a href="../ch12/ch12-01.html">
<i class="fa fa-check"></i>
<b>12.1.</b>
2015-12-21 04:55:18 +00:00
爲何需要反射?
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="12.2" data-path="ch12/ch12-02.html">
<a href="../ch12/ch12-02.html">
<i class="fa fa-check"></i>
<b>12.2.</b>
reflect.Type和reflect.Value
</a>
</li>
<li class="chapter " data-level="12.3" data-path="ch12/ch12-03.html">
<a href="../ch12/ch12-03.html">
<i class="fa fa-check"></i>
<b>12.3.</b>
Display遞歸打印
</a>
</li>
<li class="chapter " data-level="12.4" data-path="ch12/ch12-04.html">
<a href="../ch12/ch12-04.html">
<i class="fa fa-check"></i>
<b>12.4.</b>
2015-12-21 04:55:18 +00:00
示例: 編碼S表達式
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="12.5" data-path="ch12/ch12-05.html">
<a href="../ch12/ch12-05.html">
<i class="fa fa-check"></i>
<b>12.5.</b>
通過reflect.Value脩改值
</a>
</li>
<li class="chapter " data-level="12.6" data-path="ch12/ch12-06.html">
<a href="../ch12/ch12-06.html">
<i class="fa fa-check"></i>
<b>12.6.</b>
2015-12-21 04:55:18 +00:00
示例: 解碼S表達式
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="chapter " data-level="12.7" data-path="ch12/ch12-07.html">
<a href="../ch12/ch12-07.html">
<i class="fa fa-check"></i>
<b>12.7.</b>
穫取結構體字段標識
</a>
</li>
<li class="chapter " data-level="12.8" data-path="ch12/ch12-08.html">
<a href="../ch12/ch12-08.html">
<i class="fa fa-check"></i>
<b>12.8.</b>
顯示一個類型的方法集
</a>
</li>
<li class="chapter " data-level="12.9" data-path="ch12/ch12-09.html">
<a href="../ch12/ch12-09.html">
<i class="fa fa-check"></i>
<b>12.9.</b>
幾點忠告
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="13" data-path="ch13/ch13.html">
<a href="../ch13/ch13.html">
<i class="fa fa-check"></i>
<b>13.</b>
底層編程
</a>
<ul class="articles">
<li class="chapter " data-level="13.1" data-path="ch13/ch13-01.html">
<a href="../ch13/ch13-01.html">
<i class="fa fa-check"></i>
<b>13.1.</b>
unsafe.Sizeof, Alignof 和 Offsetof
</a>
</li>
<li class="chapter " data-level="13.2" data-path="ch13/ch13-02.html">
<a href="../ch13/ch13-02.html">
<i class="fa fa-check"></i>
<b>13.2.</b>
unsafe.Pointer
</a>
</li>
<li class="chapter " data-level="13.3" data-path="ch13/ch13-03.html">
<a href="../ch13/ch13-03.html">
<i class="fa fa-check"></i>
<b>13.3.</b>
示例: 深度相等判斷
</a>
</li>
<li class="chapter " data-level="13.4" data-path="ch13/ch13-04.html">
<a href="../ch13/ch13-04.html">
<i class="fa fa-check"></i>
<b>13.4.</b>
通過cgo調用C代碼
</a>
</li>
<li class="chapter " data-level="13.5" data-path="ch13/ch13-05.html">
<a href="../ch13/ch13-05.html">
<i class="fa fa-check"></i>
<b>13.5.</b>
幾點忠告
</a>
</li>
</ul>
</li>
2015-12-24 06:47:06 +00:00
<li class="chapter " data-level="14" data-path="CONTRIBUTORS.html">
2015-12-09 07:57:17 +00:00
2015-12-24 06:47:06 +00:00
<a href="../CONTRIBUTORS.html">
2015-12-09 07:57:17 +00:00
<i class="fa fa-check"></i>
2015-12-21 04:55:18 +00:00
<b>14.</b>
2015-12-09 07:57:17 +00:00
2015-12-24 06:47:06 +00:00
附録
2015-12-09 07:57:17 +00:00
</a>
</li>
<li class="divider"></li>
<li>
<a href="https://www.gitbook.com" target="blank" class="gitbook-link">
本書使用 GitBook 釋出
</a>
</li>
</ul>
</nav>
</div>
<div class="book-body">
<div class="body-inner">
<div class="book-header" role="navigation">
<!-- Actions Left -->
<!-- Title -->
<h1>
<i class="fa fa-circle-o-notch fa-spin"></i>
<a href="../" >Go编程语言</a>
</h1>
</div>
<div class="page-wrapper" tabindex="-1" role="main">
<div class="page-inner">
<section class="normal" id="section-">
2015-12-21 04:55:18 +00:00
<h2 id="86-&#x793A;&#x4F8B;-&#x4F75;&#x767C;&#x7684;web&#x722C;&#x87F2;">8.6. &#x793A;&#x4F8B;: &#x4F75;&#x767C;&#x7684;Web&#x722C;&#x87F2;</h2>
<p>&#x5728;5.6&#x7BC0;&#x4E2D;&#xFF0C;&#x6211;&#x5011;&#x505A;&#x4E86;&#x4E00;&#x500B;&#x7C21;&#x55AE;&#x7684;web&#x722C;&#x87F2;&#xFF0C;&#x7528;bfs(&#x5EE3;&#x5EA6;&#x512A;&#x5148;)&#x7B97;&#x6CD5;&#x4F86;&#x6293;&#x53D6;&#x6574;&#x500B;&#x7DB2;&#x7AD9;&#x3002;&#x5728;&#x672C;&#x7BC0;&#x4E2D;&#xFF0C;&#x6211;&#x5011;&#x6703;&#x8B93;&#x9019;&#x500B;&#x9019;&#x500B;&#x722C;&#x87F2;&#x4F75;&#x884C;&#x5316;&#xFF0C;&#x9019;&#x6A23;&#x6BCF;&#x4E00;&#x500B;&#x5F7C;&#x6B64;&#x7368;&#x7ACB;&#x7684;&#x6293;&#x53D6;&#x547D;&#x4EE4;&#x53EF;&#x4EE5;&#x4F75;&#x884C;&#x9032;&#x884C;IO&#xFF0C;&#x6700;&#x5927;&#x5316;&#x5229;&#x7528;&#x7DB2;&#x7D61;&#x8CC7;&#x6E90;&#x3002;crawl&#x51FD;&#x6578;&#x548C;gopl.io/ch5/findlinks3&#x4E2D;&#x7684;&#x662F;&#x4E00;&#x6A23;&#x7684;&#x3002;</p>
2015-12-14 04:08:47 +00:00
<pre><code class="lang-go">gopl.io/ch8/crawl1
<span class="hljs-keyword">func</span> crawl(url <span class="hljs-typename">string</span>) []<span class="hljs-typename">string</span> {
fmt.Println(url)
list, err := links.Extract(url)
<span class="hljs-keyword">if</span> err != <span class="hljs-constant">nil</span> {
log.Print(err)
}
<span class="hljs-keyword">return</span> list
}
</code></pre>
2015-12-21 04:55:18 +00:00
<p>&#x4E3B;&#x51FD;&#x6578;&#x548C;5.6&#x7BC0;&#x4E2D;&#x7684;breadthFirst(&#x6DF1;&#x5EA6;&#x512A;&#x5148;)&#x985E;&#x4F3C;&#x3002;&#x50CF;&#x4E4B;&#x524D;&#x4E00;&#x6A23;&#xFF0C;&#x4E00;&#x500B;worklist&#x662F;&#x4E00;&#x500B;&#x8A18;&#x9332;&#x4E86;&#x9700;&#x8981;&#x8655;&#x7406;&#x7684;&#x5143;&#x7D20;&#x7684;&#x968A;&#x5217;&#xFF0C;&#x6BCF;&#x4E00;&#x500B;&#x5143;&#x7D20;&#x90FD;&#x662F;&#x4E00;&#x500B;&#x9700;&#x8981;&#x6293;&#x53D6;&#x7684;URL&#x5217;&#x8868;&#xFF0C;&#x4E0D;&#x904E;&#x9019;&#x4E00;&#x6B21;&#x6211;&#x5011;&#x7528;channel&#x4EE3;&#x66FF;slice&#x4F86;&#x505A;&#x9019;&#x500B;&#x968A;&#x5217;&#x3002;&#x6BCF;&#x4E00;&#x500B;&#x5C0D;crawl&#x7684;&#x8ABF;&#x7528;&#x90FD;&#x6703;&#x5728;&#x4ED6;&#x5011;&#x81EA;&#x5DF1;&#x7684;goroutine&#x4E2D;&#x9032;&#x884C;&#x4F75;&#x4E14;&#x6703;&#x628A;&#x4ED6;&#x5011;&#x6293;&#x5230;&#x7684;&#x93C8;&#x63A5;&#x767C;&#x9001;&#x8FF4;worklist&#x3002;</p>
2015-12-14 04:08:47 +00:00
<pre><code class="lang-go"><span class="hljs-keyword">func</span> main() {
worklist := <span class="hljs-built_in">make</span>(<span class="hljs-keyword">chan</span> []<span class="hljs-typename">string</span>)
<span class="hljs-comment">// Start with the command-line arguments.</span>
<span class="hljs-keyword">go</span> <span class="hljs-keyword">func</span>() { worklist &lt;- os.Args[<span class="hljs-number">1</span>:] }()
<span class="hljs-comment">// Crawl the web concurrently.</span>
seen := <span class="hljs-built_in">make</span>(<span class="hljs-keyword">map</span>[<span class="hljs-typename">string</span>]<span class="hljs-typename">bool</span>)
<span class="hljs-keyword">for</span> list := <span class="hljs-keyword">range</span> worklist {
<span class="hljs-keyword">for</span> _, link := <span class="hljs-keyword">range</span> list {
<span class="hljs-keyword">if</span> !seen[link] {
seen[link] = <span class="hljs-constant">true</span>
<span class="hljs-keyword">go</span> <span class="hljs-keyword">func</span>(link <span class="hljs-typename">string</span>) {
worklist &lt;- crawl(link)
}(link)
}
}
}
}
</code></pre>
2015-12-21 04:55:18 +00:00
<p>&#x8A3B;&#x610F;&#x9019;&#x91CC;&#x7684;crawl&#x6240;&#x5728;&#x7684;goroutine&#x6703;&#x5C07;link&#x4F5C;&#x7232;&#x4E00;&#x500B;&#x986F;&#x5F0F;&#x7684;&#x53C3;&#x6578;&#x50B3;&#x5165;&#xFF0C;&#x4F86;&#x907F;&#x514D;&#x201C;&#x5FAA;&#x74B0;&#x8B8A;&#x91CF;&#x5FEB;&#x7167;&#x201D;&#x7684;&#x554F;&#x984C;(&#x5728;5.6.1&#x4E2D;&#x6709;&#x8B1B;&#x89E3;)&#x3002;&#x53E6;&#x5916;&#x8A3B;&#x610F;&#x9019;&#x91CC;&#x5C07;&#x547D;&#x4EE4;&#x884C;&#x53C3;&#x6578;&#x50B3;&#x5165;worklist&#x4E5F;&#x662F;&#x5728;&#x4E00;&#x500B;&#x53E6;&#x5916;&#x7684;goroutine&#x4E2D;&#x9032;&#x884C;&#x7684;&#xFF0C;&#x9019;&#x662F;&#x7232;&#x4E86;&#x907F;&#x514D;&#x5728;main goroutine&#x548C;crawler goroutine&#x4E2D;&#x540C;&#x6642;&#x5411;&#x53E6;&#x4E00;&#x500B;goroutine&#x901A;&#x904E;channel&#x767C;&#x9001;&#x5167;&#x5BB9;&#x6642;&#x767C;&#x751F;&#x6B7B;&#x9396;(&#x56E0;&#x7232;&#x53E6;&#x4E00;&#x908A;&#x7684;&#x63A5;&#x6536;&#x64CD;&#x4F5C;&#x9084;&#x6C92;&#x6709;&#x6E96;&#x5099;&#x597D;)&#x3002;&#x7576;&#x7136;&#xFF0C;&#x9019;&#x91CC;&#x6211;&#x5011;&#x4E5F;&#x53EF;&#x4EE5;&#x7528;buffered channel&#x4F86;&#x89E3;&#x6C7A;&#x554F;&#x984C;&#xFF0C;&#x9019;&#x91CC;&#x4E0D;&#x518D;&#x8D05;&#x8FF0;&#x3002;</p>
<p>&#x73FE;&#x5728;&#x722C;&#x87F2;&#x53EF;&#x4EE5;&#x9AD8;&#x4F75;&#x767C;&#x5730;&#x904B;&#x884C;&#x8D77;&#x4F86;&#xFF0C;&#x4F75;&#x4E14;&#x53EF;&#x4EE5;&#x7523;&#x751F;&#x4E00;&#x5927;&#x5768;&#x7684;URL&#x4E86;&#xFF0C;&#x4E0D;&#x904E;&#x9084;&#x662F;&#x6703;&#x6709;&#x5006;&#x554F;&#x984C;&#x3002;&#x4E00;&#x500B;&#x554F;&#x984C;&#x662F;&#x5728;&#x904B;&#x884C;&#x4E00;&#x6BB5;&#x6642;&#x9593;&#x5F8C;&#x53EF;&#x80FD;&#x6703;&#x9F63;&#x73FE;&#x5728;log&#x7684;&#x932F;&#x8AA4;&#x4FE1;&#x606F;&#x91CC;&#x7684;&#xFF1A;</p>
2015-12-14 04:08:47 +00:00
<pre><code>$ go build gopl.io/ch8/crawl1
$ ./crawl1 http://gopl.io/
http://gopl.io/
https://golang.org/help/
https://golang.org/doc/
https://golang.org/blog/
...
2015/07/15 18:22:12 Get ...: dial tcp: lookup blog.golang.org: no such host
2015/07/15 18:22:12 Get ...: dial tcp 23.21.222.120:443: socket:
too many open files
...
2015-12-21 04:55:18 +00:00
</code></pre><p>&#x6700;&#x521D;&#x7684;&#x932F;&#x8AA4;&#x4FE1;&#x606F;&#x662F;&#x4E00;&#x500B;&#x8B93;&#x4EBA;&#x83AB;&#x540D;&#x7684;DNS&#x67FB;&#x627E;&#x5931;&#x6557;&#xFF0C;&#x537D;&#x4F7F;&#x9019;&#x500B;&#x57DF;&#x540D;&#x662F;&#x5B8C;&#x5168;&#x53EF;&#x9760;&#x7684;&#x3002;&#x800C;&#x96A8;&#x5F8C;&#x7684;&#x932F;&#x8AA4;&#x4FE1;&#x606F;&#x63ED;&#x793A;&#x4E86;&#x539F;&#x56E0;&#xFF1A;&#x9019;&#x500B;&#x7A0B;&#x5E8F;&#x4E00;&#x6B21;&#x6027;&#x5275;&#x5EFA;&#x4E86;&#x592A;&#x591A;&#x7DB2;&#x7D61;&#x9023;&#x63A5;&#xFF0C;&#x8D85;&#x904E;&#x4E86;&#x6BCF;&#x4E00;&#x500B;&#x9032;&#x7A0B;&#x7684;&#x6253;&#x958B;&#x6587;&#x4EF6;&#x6578;&#x9650;&#x88FD;&#xFF0C;&#x65E3;&#x800C;&#x5C0E;&#x81F4;&#x4E86;&#x5728;&#x8ABF;&#x7528;net.Dial&#x50CF;DNS&#x67FB;&#x627E;&#x5931;&#x6557;&#x9019;&#x6A23;&#x7684;&#x554F;&#x984C;&#x3002;</p>
<p>&#x9019;&#x500B;&#x7A0B;&#x5E8F;&#x5BE6;&#x5728;&#x662F;&#x592A;&#x4ED6;&#x5ABD;&#x4F75;&#x884C;&#x4E86;&#x3002;&#x7121;&#x7AAE;&#x7121;&#x76E1;&#x5730;&#x4F75;&#x884C;&#x5316;&#x4F75;&#x4E0D;&#x662F;&#x4EC0;&#x9EBD;&#x597D;&#x4E8B;&#x60C5;&#xFF0C;&#x56E0;&#x7232;&#x4E0D;&#x7BA1;&#x600E;&#x9EBD;&#x8AAC;&#xFF0C;&#x4F60;&#x7684;&#x7E6B;&#x7D71;&#x7E3D;&#x662F;&#x6703;&#x6709;&#x4E00;&#x500B;&#x4E9B;&#x9650;&#x88FD;&#x56E0;&#x7D20;&#xFF0C;&#x6BD4;&#x5982;CPU&#x8988;&#x5FC3;&#x6578;&#x6703;&#x9650;&#x88FD;&#x4F60;&#x7684;&#x8A08;&#x7B97;&#x8CA0;&#x8F09;&#xFF0C;&#x6BD4;&#x5982;&#x4F60;&#x7684;&#x786C;&#x76E4;&#x8F49;&#x8EF8;&#x548C;&#x78C1;&#x982D;&#x6578;&#x9650;&#x88FD;&#x4E86;&#x4F60;&#x7684;&#x672C;&#x5730;&#x78C1;&#x76E4;IO&#x64CD;&#x4F5C;&#x983B;&#x7387;&#xFF0C;&#x6BD4;&#x5982;&#x4F60;&#x7684;&#x7DB2;&#x7D61;&#x5E36;&#x5BEC;&#x9650;&#x88FD;&#x4E86;&#x4F60;&#x7684;&#x4E0B;&#x8F09;&#x901F;&#x5EA6;&#x4E0A;&#x9650;&#xFF0C;&#x6216;&#x8005;&#x662F;&#x4F60;&#x7684;&#x4E00;&#x500B;web&#x670D;&#x52D9;&#x7684;&#x670D;&#x52D9;&#x5BB9;&#x91CF;&#x4E0A;&#x9650;&#x7B49;&#x7B49;&#x3002;&#x7232;&#x4E86;&#x89E3;&#x6C7A;&#x9019;&#x500B;&#x554F;&#x984C;&#xFF0C;&#x6211;&#x5011;&#x53EF;&#x4EE5;&#x9650;&#x88FD;&#x4F75;&#x767C;&#x7A0B;&#x5E8F;&#x6240;&#x4F7F;&#x7528;&#x7684;&#x8CC7;&#x6E90;&#x4F86;&#x4F7F;&#x4E4B;&#x9069;&#x61C9;&#x81EA;&#x5DF1;&#x7684;&#x904B;&#x884C;&#x74B0;&#x5883;&#x3002;&#x5C0D;&#x65BC;&#x6211;&#x5011;&#x7684;&#x4F8B;&#x5B50;&#x4F86;&#x8AAC;&#xFF0C;&#x6700;&#x7C21;&#x55AE;&#x7684;&#x65B9;&#x6CD5;&#x5C31;&#x662F;&#x9650;&#x88FD;&#x5C0D;links.Extract&#x5728;&#x540C;&#x4E00;&#x6642;&#x9593;&#x6700;&#x591A;&#x4E0D;&#x6703;&#x6709;&#x8D85;&#x904E;n&#x6B21;&#x8ABF;&#x7528;&#xFF0C;&#x9019;&#x91CC;&#x7684;n&#x662F;fd&#x7684;limit-20&#xFF0C;&#x4E00;&#x822C;&#x60C5;&#x6CC1;&#x4E0B;&#x3002;&#x9019;&#x500B;&#x4E00;&#x500B;&#x591C;&#x5E97;&#x91CC;&#x9650;&#x88FD;&#x5BA2;&#x4EBA;&#x6578;&#x76EE;&#x662F;&#x4E00;&#x500B;&#x9053;&#x7406;&#xFF0C;&#x96BB;&#x6709;&#x7576;&#x6709;&#x5BA2;&#x4EBA;&#x96E2;&#x958B;&#x6642;&#xFF0C;&#x7E94;&#x6703;&#x5141;&#x8A31;&#x65B0;&#x7684;&#x5BA2;&#x4EBA;&#x9032;&#x5165;&#x5E97;&#x5167;(&#x8B6F;&#x8A3B;&#xFF1A;&#x4F5C;&#x8005;&#x4F60;&#x500B;&#x8001;&#x6D41;&#x6C13;)&#x3002;</p>
<p>&#x6211;&#x5011;&#x53EF;&#x4EE5;&#x7528;&#x4E00;&#x500B;&#x6709;&#x5BB9;&#x91CF;&#x9650;&#x88FD;&#x7684;buffered channel&#x4F86;&#x63A7;&#x88FD;&#x4F75;&#x767C;&#xFF0C;&#x9019;&#x985E;&#x4F3C;&#x65BC;&#x64CD;&#x4F5C;&#x7E6B;&#x7D71;&#x91CC;&#x7684;&#x8A08;&#x6578;&#x4FE1;&#x865F;&#x91CF;&#x6982;&#x5FF5;&#x3002;&#x5F9E;&#x6982;&#x5FF5;&#x4E0A;&#x8B1B;&#xFF0C;channel&#x91CC;&#x7684;n&#x500B;&#x7A7A;&#x69FD;&#x4EE3;&#x8868;n&#x500B;&#x53EF;&#x4EE5;&#x8655;&#x7406;&#x5167;&#x5BB9;&#x7684;token(&#x901A;&#x884C;&#x8B49;)&#xFF0C;&#x5F9E;channel&#x91CC;&#x63A5;&#x6536;&#x4E00;&#x500B;&#x503C;&#x6703;&#x91CB;&#x653E;&#x5176;&#x4E2D;&#x7684;&#x4E00;&#x500B;token&#xFF0C;&#x4F75;&#x4E14;&#x751F;&#x6210;&#x4E00;&#x500B;&#x65B0;&#x7684;&#x7A7A;&#x69FD;&#x4F4D;&#x3002;&#x9019;&#x6A23;&#x4FDD;&#x8B49;&#x4E86;&#x5728;&#x6C92;&#x6709;&#x63A5;&#x6536;&#x4ECB;&#x5165;&#x6642;&#x6700;&#x591A;&#x6709;n&#x500B;&#x767C;&#x9001;&#x64CD;&#x4F5C;&#x3002;(&#x9019;&#x91CC;&#x53EF;&#x80FD;&#x6211;&#x5011;&#x62FF;channel&#x91CC;&#x586B;&#x5145;&#x7684;&#x69FD;&#x4F86;&#x505A;token&#x66F4;&#x76F4;&#x89C0;&#x4E00;&#x4E9B;&#xFF0C;&#x4E0D;&#x904E;&#x9084;&#x662F;&#x9019;&#x6A23;&#x5427;~)&#x3002;&#x7531;&#x65BC;channel&#x91CC;&#x7684;&#x5143;&#x7D20;&#x985E;&#x578B;&#x4F75;&#x4E0D;&#x91CD;&#x8981;&#xFF0C;&#x6211;&#x5011;&#x7528;&#x4E00;&#x500B;&#x96F6;&#x503C;&#x7684;struct{}&#x4F86;&#x4F5C;&#x7232;&#x5176;&#x5143;&#x7D20;&#x3002;</p>
<p>&#x8B93;&#x6211;&#x5011;&#x91CD;&#x5BEB;crawl&#x51FD;&#x6578;&#xFF0C;&#x5C07;&#x5C0D;links.Extract&#x7684;&#x8ABF;&#x7528;&#x64CD;&#x4F5C;&#x7528;&#x7A6B;&#x53D6;&#x3001;&#x91CB;&#x653E;token&#x7684;&#x64CD;&#x4F5C;&#x5305;&#x88F9;&#x8D77;&#x4F86;&#xFF0C;&#x4F86;&#x78BA;&#x4FDD;&#x540C;&#x4E00;&#x6642;&#x9593;&#x5C0D;&#x5176;&#x96BB;&#x6709;20&#x500B;&#x8ABF;&#x7528;&#x3002;&#x4FE1;&#x865F;&#x91CF;&#x6578;&#x91CF;&#x548C;&#x5176;&#x80FD;&#x64CD;&#x4F5C;&#x7684;IO&#x8CC7;&#x6E90;&#x6578;&#x91CF;&#x61C9;&#x4FDD;&#x6301;&#x63A5;&#x8FD1;&#x3002;</p>
2015-12-14 04:08:47 +00:00
<pre><code class="lang-go">gopl.io/ch8/crawl2
<span class="hljs-comment">// tokens is a counting semaphore used to</span>
<span class="hljs-comment">// enforce a limit of 20 concurrent requests.</span>
<span class="hljs-keyword">var</span> tokens = <span class="hljs-built_in">make</span>(<span class="hljs-keyword">chan</span> <span class="hljs-keyword">struct</span>{}, <span class="hljs-number">20</span>)
<span class="hljs-keyword">func</span> crawl(url <span class="hljs-typename">string</span>) []<span class="hljs-typename">string</span> {
fmt.Println(url)
tokens &lt;- <span class="hljs-keyword">struct</span>{}{} <span class="hljs-comment">// acquire a token</span>
list, err := links.Extract(url)
&lt;-tokens <span class="hljs-comment">// release the token</span>
<span class="hljs-keyword">if</span> err != <span class="hljs-constant">nil</span> {
log.Print(err)
}
<span class="hljs-keyword">return</span> list
}
</code></pre>
2015-12-21 04:55:18 +00:00
<p>&#x7B2C;&#x4E8C;&#x500B;&#x554F;&#x984C;&#x662F;&#x9019;&#x500B;&#x7A0B;&#x5E8F;&#x6C38;&#x9060;&#x90FD;&#x4E0D;&#x6703;&#x7D42;&#x6B62;&#xFF0C;&#x537D;&#x4F7F;&#x5B83;&#x5DF2;&#x7D93;&#x722C;&#x5230;&#x4E86;&#x6240;&#x6709;&#x521D;&#x59CB;&#x93C8;&#x63A5;&#x884D;&#x751F;&#x9F63;&#x7684;&#x93C8;&#x63A5;&#x3002;(&#x7576;&#x7136;&#xFF0C;&#x9664;&#x975E;&#x4F60;&#x614E;&#x91CD;&#x5730;&#x9078;&#x64C7;&#x4E86;&#x5408;&#x9069;&#x7684;&#x521D;&#x59CB;&#x5316;URL&#x6216;&#x8005;&#x5DF2;&#x7D93;&#x5BE6;&#x73FE;&#x4E86;&#x7DF4;&#x7FD2;8.6&#x4E2D;&#x7684;&#x6DF1;&#x5EA6;&#x9650;&#x88FD;&#xFF0C;&#x4F60;&#x61C9;&#x8A72;&#x9084;&#x6C92;&#x6709;&#x610F;&#x8B58;&#x5230;&#x9019;&#x500B;&#x554F;&#x984C;)&#x3002;&#x7232;&#x4E86;&#x4F7F;&#x9019;&#x500B;&#x7A0B;&#x5E8F;&#x80FD;&#x5920;&#x7D42;&#x6B62;&#xFF0C;&#x6211;&#x5011;&#x9700;&#x8981;&#x5728;worklist&#x7232;&#x7A7A;&#x6216;&#x8005;&#x6C92;&#x6709;crawl&#x7684;goroutine&#x5728;&#x904B;&#x884C;&#x6642;&#x9000;&#x9F63;&#x4E3B;&#x5FAA;&#x74B0;&#x3002;</p>
2015-12-14 04:08:47 +00:00
<pre><code class="lang-go"><span class="hljs-keyword">func</span> main() {
worklist := <span class="hljs-built_in">make</span>(<span class="hljs-keyword">chan</span> []<span class="hljs-typename">string</span>)
<span class="hljs-keyword">var</span> n <span class="hljs-typename">int</span> <span class="hljs-comment">// number of pending sends to worklist</span>
<span class="hljs-comment">// Start with the command-line arguments.</span>
n++
<span class="hljs-keyword">go</span> <span class="hljs-keyword">func</span>() { worklist &lt;- os.Args[<span class="hljs-number">1</span>:] }()
<span class="hljs-comment">// Crawl the web concurrently.</span>
seen := <span class="hljs-built_in">make</span>(<span class="hljs-keyword">map</span>[<span class="hljs-typename">string</span>]<span class="hljs-typename">bool</span>)
<span class="hljs-keyword">for</span> ; n &gt; <span class="hljs-number">0</span>; n-- {
list := &lt;-worklist
<span class="hljs-keyword">for</span> _, link := <span class="hljs-keyword">range</span> list {
<span class="hljs-keyword">if</span> !seen[link] {
seen[link] = <span class="hljs-constant">true</span>
n++
<span class="hljs-keyword">go</span> <span class="hljs-keyword">func</span>(link <span class="hljs-typename">string</span>) {
worklist &lt;- crawl(link)
}(link)
}
}
}
}
</code></pre>
2015-12-21 04:55:18 +00:00
<p>&#x9019;&#x500B;&#x7248;&#x672C;&#x4E2D;&#xFF0C;&#x8A08;&#x7B97;&#x5668;n&#x5C0D;worklist&#x7684;&#x767C;&#x9001;&#x64CD;&#x4F5C;&#x6578;&#x91CF;&#x9032;&#x884C;&#x4E86;&#x9650;&#x88FD;&#x3002;&#x6BCF;&#x4E00;&#x6B21;&#x6211;&#x5011;&#x767C;&#x73FE;&#x6709;&#x5143;&#x7D20;&#x9700;&#x8981;&#x88AB;&#x767C;&#x9001;&#x5230;worklist&#x6642;&#xFF0C;&#x6211;&#x5011;&#x90FD;&#x6703;&#x5C0D;n&#x9032;&#x884C;++&#x64CD;&#x4F5C;&#xFF0C;&#x5728;&#x5411;worklist&#x4E2D;&#x767C;&#x9001;&#x521D;&#x59CB;&#x7684;&#x547D;&#x4EE4;&#x884C;&#x53C3;&#x6578;&#x4E4B;&#x524D;&#xFF0C;&#x6211;&#x5011;&#x4E5F;&#x9032;&#x884C;&#x904E;&#x4E00;&#x6B21;++&#x64CD;&#x4F5C;&#x3002;&#x9019;&#x91CC;&#x7684;&#x64CD;&#x4F5C;++&#x662F;&#x5728;&#x6BCF;&#x555F;&#x52D5;&#x4E00;&#x500B;crawler&#x7684;goroutine&#x4E4B;&#x524D;&#x3002;&#x4E3B;&#x5FAA;&#x74B0;&#x6703;&#x5728;n&#x6E1B;&#x7232;0&#x6642;&#x7D42;&#x6B62;&#xFF0C;&#x9019;&#x6642;&#x5019;&#x8AAC;&#x660E;&#x6C92;&#x6D3B;&#x53EF;&#x69A6;&#x4E86;&#x3002;</p>
<p>&#x73FE;&#x5728;&#x9019;&#x500B;&#x4F75;&#x767C;&#x722C;&#x87F2;&#x6703;&#x6BD4;5.6&#x7BC0;&#x4E2D;&#x7684;&#x6DF1;&#x5EA6;&#x512A;&#x5148;&#x8490;&#x7D22;&#x7248;&#x5FEB;&#x4E0A;20&#x500D;&#xFF0C;&#x800C;&#x4E14;&#x4E0D;&#x6703;&#x9F63;&#x4EC0;&#x9EBD;&#x932F;&#xFF0C;&#x4F75;&#x4E14;&#x5728;&#x5176;&#x5B8C;&#x6210;&#x4EFB;&#x52D9;&#x6642;&#x4E5F;&#x6703;&#x6B63;&#x78BA;&#x5730;&#x7D42;&#x6B62;&#x3002;</p>
<p>&#x4E0B;&#x9762;&#x7684;&#x7A0B;&#x5E8F;&#x662F;&#x907F;&#x514D;&#x904E;&#x5EA6;&#x4F75;&#x767C;&#x7684;&#x53E6;&#x4E00;&#x7A2E;&#x601D;&#x8DEF;&#x3002;&#x9019;&#x500B;&#x7248;&#x672C;&#x4F7F;&#x7528;&#x4E86;&#x539F;&#x4F86;&#x7684;crawl&#x51FD;&#x6578;&#xFF0C;&#x4F46;&#x6C92;&#x6709;&#x4F7F;&#x7528;&#x8A08;&#x6578;&#x4FE1;&#x865F;&#x91CF;&#xFF0C;&#x53D6;&#x800C;&#x4EE3;&#x4E4B;&#x7528;&#x4E86;20&#x500B;&#x9577;&#x6D3B;&#x7684;crawler goroutine&#xFF0C;&#x9019;&#x6A23;&#x4F86;&#x4FDD;&#x8B49;&#x6700;&#x591A;20&#x500B;HTTP&#x8ACB;&#x6C42;&#x5728;&#x4F75;&#x767C;&#x3002;</p>
2015-12-14 04:08:47 +00:00
<pre><code class="lang-go"><span class="hljs-keyword">func</span> main() {
worklist := <span class="hljs-built_in">make</span>(<span class="hljs-keyword">chan</span> []<span class="hljs-typename">string</span>) <span class="hljs-comment">// lists of URLs, may have duplicates</span>
unseenLinks := <span class="hljs-built_in">make</span>(<span class="hljs-keyword">chan</span> <span class="hljs-typename">string</span>) <span class="hljs-comment">// de-duplicated URLs</span>
<span class="hljs-comment">// Add command-line arguments to worklist.</span>
<span class="hljs-keyword">go</span> <span class="hljs-keyword">func</span>() { worklist &lt;- os.Args[<span class="hljs-number">1</span>:] }()
<span class="hljs-comment">// Create 20 crawler goroutines to fetch each unseen link.</span>
<span class="hljs-keyword">for</span> i := <span class="hljs-number">0</span>; i &lt; <span class="hljs-number">20</span>; i++ {
<span class="hljs-keyword">go</span> <span class="hljs-keyword">func</span>() {
<span class="hljs-keyword">for</span> link := <span class="hljs-keyword">range</span> unseenLinks {
foundLinks := crawl(link)
<span class="hljs-keyword">go</span> <span class="hljs-keyword">func</span>() { worklist &lt;- foundLinks }()
}
}()
}
<span class="hljs-comment">// The main goroutine de-duplicates worklist items</span>
<span class="hljs-comment">// and sends the unseen ones to the crawlers.</span>
seen := <span class="hljs-built_in">make</span>(<span class="hljs-keyword">map</span>[<span class="hljs-typename">string</span>]<span class="hljs-typename">bool</span>)
<span class="hljs-keyword">for</span> list := <span class="hljs-keyword">range</span> worklist {
<span class="hljs-keyword">for</span> _, link := <span class="hljs-keyword">range</span> list {
<span class="hljs-keyword">if</span> !seen[link] {
seen[link] = <span class="hljs-constant">true</span>
unseenLinks &lt;- link
}
}
}
}
</code></pre>
2015-12-21 04:55:18 +00:00
<p>&#x6240;&#x6709;&#x7684;&#x722C;&#x87F2;goroutine&#x73FE;&#x5728;&#x90FD;&#x662F;&#x88AB;&#x540C;&#x4E00;&#x500B;channel-unseenLinks&#x9935;&#x98FD;&#x7684;&#x4E86;&#x3002;&#x4E3B;goroutine&#x8CA0;&#x8CAC;&#x62C6;&#x5206;&#x5B83;&#x5F9E;worklist&#x91CC;&#x62FF;&#x5230;&#x7684;&#x5143;&#x7D20;&#xFF0C;&#x7136;&#x5F8C;&#x628A;&#x6C92;&#x6709;&#x6293;&#x904E;&#x7684;&#x7D93;&#x7531;unseenLinks channel&#x767C;&#x9001;&#x7D66;&#x4E00;&#x500B;&#x722C;&#x87F2;&#x7684;goroutine&#x3002;</p>
<p>seen&#x9019;&#x500B;map&#x88AB;&#x9650;&#x5B9A;&#x5728;main goroutine&#x4E2D;&#xFF1B;&#x4E5F;&#x5C31;&#x662F;&#x8AAC;&#x9019;&#x500B;map&#x96BB;&#x80FD;&#x5728;main goroutine&#x4E2D;&#x9032;&#x884C;&#x8A2A;&#x554F;&#x3002;&#x985E;&#x4F3C;&#x65BC;&#x5176;&#x5B83;&#x7684;&#x4FE1;&#x606F;&#x96B1;&#x85CF;&#x65B9;&#x5F0F;&#xFF0C;&#x9019;&#x6A23;&#x7684;&#x7D04;&#x675F;&#x53EF;&#x4EE5;&#x8B93;&#x6211;&#x5011;&#x5F9E;&#x4E00;&#x5B9A;&#x7A0B;&#x5EA6;&#x4E0A;&#x4FDD;&#x8B49;&#x7A0B;&#x5E8F;&#x7684;&#x6B63;&#x78BA;&#x6027;&#x3002;&#x4F8B;&#x5982;&#xFF0C;&#x5167;&#x90E8;&#x8B8A;&#x91CF;&#x4E0D;&#x80FD;&#x5920;&#x5728;&#x51FD;&#x6578;&#x5916;&#x90E8;&#x88AB;&#x8A2A;&#x554F;&#x5230;&#xFF1B;&#x8B8A;&#x91CF;(&#xA7;2.3.4)&#x5728;&#x6C92;&#x6709;&#x88AB;&#x8F49;&#x7FA9;&#x7684;&#x60C5;&#x6CC1;&#x4E0B;&#x662F;&#x7121;&#x6CD5;&#x5728;&#x51FD;&#x6578;&#x5916;&#x90E8;&#x8A2A;&#x554F;&#x7684;&#xFF1B;&#x4E00;&#x500B;&#x5C0D;&#x8C61;&#x7684;&#x5C01;&#x88DD;&#x5B57;&#x6BB5;&#x7121;&#x6CD5;&#x88AB;&#x8A72;&#x5C0D;&#x8C61;&#x7684;&#x65B9;&#x6CD5;&#x4EE5;&#x5916;&#x7684;&#x65B9;&#x6CD5;&#x8A2A;&#x554F;&#x5230;&#x3002;&#x5728;&#x6240;&#x6709;&#x7684;&#x60C5;&#x6CC1;&#x4E0B;&#xFF0C;&#x4FE1;&#x606F;&#x96B1;&#x85CF;&#x90FD;&#x53EF;&#x4EE5;&#x5E6B;&#x52A9;&#x6211;&#x5011;&#x7D04;&#x675F;&#x6211;&#x5011;&#x7684;&#x7A0B;&#x5E8F;&#xFF0C;&#x4F7F;&#x5176;&#x4E0D;&#x767C;&#x751F;&#x610F;&#x6599;&#x4E4B;&#x5916;&#x7684;&#x60C5;&#x6CC1;&#x3002;</p>
<p>crawl&#x51FD;&#x6578;&#x722C;&#x5230;&#x7684;&#x93C8;&#x63A5;&#x5728;&#x4E00;&#x500B;&#x5C08;&#x6709;&#x7684;goroutine&#x4E2D;&#x88AB;&#x767C;&#x9001;&#x5230;worklist&#x4E2D;&#x4F86;&#x907F;&#x514D;&#x6B7B;&#x9396;&#x3002;&#x7232;&#x4E86;&#x7BC0;&#x7701;&#x7A7A;&#x9593;&#xFF0C;&#x9019;&#x500B;&#x4F8B;&#x5B50;&#x7684;&#x7D42;&#x6B62;&#x554F;&#x984C;&#x6211;&#x5011;&#x5148;&#x4E0D;&#x9032;&#x884C;&#x8A73;&#x7D30;&#x95E1;&#x8FF0;&#x4E86;&#x3002;</p>
<p>&#x7DF4;&#x7FD2;8.6: &#x7232;&#x4F75;&#x767C;&#x722C;&#x87F2;&#x589E;&#x52A0;&#x6DF1;&#x5EA6;&#x9650;&#x88FD;&#x3002;&#x4E5F;&#x5C31;&#x662F;&#x8AAC;&#xFF0C;&#x5982;&#x679C;&#x7528;&#x6236;&#x8A2D;&#x7F6E;&#x4E86;depth=3&#xFF0C;&#x90A3;&#x9EBD;&#x96BB;&#x6709;&#x5F9E;&#x9996;&#x9801;&#x8DF3;&#x8F49;&#x4E09;&#x6B21;&#x4EE5;&#x5167;&#x80FD;&#x5920;&#x8DF3;&#x5230;&#x7684;&#x9801;&#x9762;&#x7E94;&#x80FD;&#x88AB;&#x6293;&#x53D6;&#x5230;&#x3002;</p>
<p>&#x7DF4;&#x7FD2;8.7: &#x5B8C;&#x6210;&#x4E00;&#x500B;&#x4F75;&#x767C;&#x7A0B;&#x5E8F;&#x4F86;&#x5275;&#x5EFA;&#x4E00;&#x500B;&#x7DDA;&#x4E0A;&#x7DB2;&#x7AD9;&#x7684;&#x672C;&#x5730;&#x93E1;&#x50CF;&#xFF0C;&#x628A;&#x8A72;&#x7AD9;&#x9EDE;&#x7684;&#x6240;&#x6709;&#x53EF;&#x9054;&#x7684;&#x9801;&#x9762;&#x90FD;&#x6293;&#x53D6;&#x5230;&#x672C;&#x5730;&#x786C;&#x76E4;&#x3002;&#x7232;&#x4E86;&#x7701;&#x4E8B;&#xFF0C;&#x6211;&#x5011;&#x9019;&#x91CC;&#x53EF;&#x4EE5;&#x96BB;&#x53D6;&#x9F63;&#x73FE;&#x5728;&#x8A72;&#x57DF;&#x4E0B;&#x7684;&#x6240;&#x6709;&#x9801;&#x9762;(&#x6BD4;&#x5982;golang.org&#x7D50;&#x5C3E;&#xFF0C;&#x8B6F;&#x8A3B;&#xFF1A;&#x5916;&#x93C8;&#x7684;&#x61C9;&#x8A72;&#x5C31;&#x4E0D;&#x7B97;&#x4E86;&#x3002;)&#x7576;&#x7136;&#x4E86;&#xFF0C;&#x9F63;&#x73FE;&#x5728;&#x9801;&#x9762;&#x91CC;&#x7684;&#x93C8;&#x63A5;&#x4F60;&#x4E5F;&#x9700;&#x8981;&#x9032;&#x884C;&#x4E00;&#x4E9B;&#x8655;&#x7406;&#xFF0C;&#x4F7F;&#x5176;&#x80FD;&#x5920;&#x5728;&#x4F60;&#x7684;&#x93E1;&#x50CF;&#x7AD9;&#x9EDE;&#x4E0A;&#x9032;&#x884C;&#x8DF3;&#x8F49;&#xFF0C;&#x800C;&#x4E0D;&#x662F;&#x6307;&#x5411;&#x539F;&#x59CB;&#x7684;&#x93C8;&#x63A5;&#x3002;</p>
2015-12-14 04:08:47 +00:00
<p>&#x8B6F;&#x8A3B;&#xFF1A;
2015-12-21 04:55:18 +00:00
&#x62D3;&#x5C55;&#x95B2;&#x8B80;&#xFF1A;
2015-12-14 04:08:47 +00:00
<a href="http://marcio.io/2015/07/handling-1-million-requests-per-minute-with-golang/" target="_blank">http://marcio.io/2015/07/handling-1-million-requests-per-minute-with-golang/</a></p>
2015-12-09 07:57:17 +00:00
</section>
</div>
</div>
</div>
<a href="../ch8/ch8-05.html" class="navigation navigation-prev " aria-label="Previous page: 併行的循環"><i class="fa fa-angle-left"></i></a>
2015-12-21 04:55:18 +00:00
<a href="../ch8/ch8-07.html" class="navigation navigation-next " aria-label="Next page: 基於select的多路複用"><i class="fa fa-angle-right"></i></a>
2015-12-09 07:57:17 +00:00
</div>
</div>
<script src="../gitbook/app.js"></script>
<script src="../gitbook/plugins/gitbook-plugin-search/lunr.min.js"></script>
<script src="../gitbook/plugins/gitbook-plugin-search/search.js"></script>
<script src="../gitbook/plugins/gitbook-plugin-sharing/buttons.js"></script>
<script src="../gitbook/plugins/gitbook-plugin-fontsettings/buttons.js"></script>
<script>
require(["gitbook"], function(gitbook) {
var config = {"highlight":{},"search":{},"sharing":{"facebook":true,"twitter":true,"google":false,"weibo":false,"instapaper":false,"vk":false,"all":["facebook","google","twitter","weibo","instapaper"]},"fontsettings":{"theme":"white","family":"sans","size":2}};
gitbook.start(config);
});
</script>
</body>
</html>