发行 1.9版本

CHANGELOG.md

@@ -1,3 +1,21 @@
+### Release v1.9
+
+2019.04.03  
+
+- 优化 文件夹监听效率，减少大量深层文件夹遍历的时间  
+- 优化 代码逻辑  
+- 增加 readme 英文说明  
+- 修复 偶现监听项目主目录无效的问题  
+
+
+### Release v1.8
+
+2019.02.27  
+
+- 使用 go1.12 编译  
+
+
+
 ### Release v1.7
 
 2019.01.24  

README.md

@@ -1,8 +1,11 @@
 ## 项目说明  
 
 fileboy，文件变更监听通知系统，使用 Go 编写。  
-
 适用于 Hot Reload （典型的如开发go项目，无需每次手动执行 go build；又比如前端 node 打包） 或者 系统监控的场景。  
+___  
+Fileboy, File Change Monitoring Notification System, written with Go.  
+For Hot Reload scenarios (typically for developing go projects without having to perform go build manually every time; for example, front-end node packaging) or system monitoring.  
+
 
 ## 特性  
 
@@ -14,10 +17,20 @@ fileboy，文件变更监听通知系统，使用 Go 编写。
 - 支持冗余任务丢弃，自定义冗余任务范围  
 - 支持 http 通知  
 - 更多...  
+___  
+- Minimalist usage and configuration  
+- Support multiple platforms, Windows/Linux/MacOS  
+- Supports custom file listening scope, listening for specified folders/not listening for specified folders/specified suffix files  
+- Support for setting up multiple commands  
+- Command support variable placeholders  
+- Supporting redundant task discarding and customizing redundant task scope  
+- Supporting HTTP notifications  
+- more...  
+
 
 ## 编译环境    
 
-go version >=1.10，推荐 1.11   
+go version 1.12   
 
 ## 更新日志  
 
@@ -28,15 +41,16 @@ go version >=1.10，推荐 1.11
 
 ### 下载二进制文件   
 
-Github: [正式版 v1.7](https://github.com/dengsgo/fileboy/releases)  
-Gitee:  [正式版 v1.7](https://gitee.com/dengsgo/fileboy/releases)  
+Github: [download v1.9](https://github.com/dengsgo/fileboy/releases)  
+Gitee:  [dowmload v1.9](https://gitee.com/dengsgo/fileboy/releases)  
 
 下载已经编译好的对应平台二进制文件，重命名为`fileboy`, 加入系统 Path 中即可。 
+___  
+Download the compiled binary file of the corresponding platform, rename it `fileboy', and add it to the system Path.  
 
 ### 源码编译   
 
-clone 该项目，进入主目录，运行命令:    
-
+clone 该项目，进入主目录，运行命令:  
 ```shell
 ## 安装依赖
 go get -u gopkg.in/fsnotify/fsnotify.v1
@@ -46,22 +60,36 @@ go build
 ## 运行
 ./fileboy
 ```
-
+___  
+Clone project, enter the project directory, run the command:  
+```shell
+## installation dependency
+go get-u gopkg.in/fsnotify/fsnotify.v1
+go get-u gopkg.in/yaml.v2
+## compile
+go build
+## run
+./fileboy
+```
 ## 使用
 
 fileboy 的正常运行依赖于 `filegirl.yaml` 配置，所以首次在项目中使用需要初始化 `filegirl.yaml`。  
-
 - 进入你想要 hot reload 的项目主目录下；  
-
 - 运行 `fileboy init`，会在该目录下生成 `filegirl.yaml`文件；  
-
 - 查看 `filegirl.yaml`,修改为适合自己项目的配置项；  
-
 - 运行 `fileboy`即可.  
-
-  如果你定义了 `command -> exec`命令，想事先确认是否能正常执行，可以运行 `fileboy exec`命令，系统会尝试运行你的自定义命令。  
-
+  
+如果你定义了 `command -> exec`命令，想事先确认是否能正常执行，可以运行 `fileboy exec`命令，系统会尝试运行你的自定义命令。  
 你可以使用 `fileboy help`查看使用帮助。  
+___  
+The normal operation of fileboy depends on the `filegirl.yaml` configuration, so for the first time in a project, `filegirl.yaml` needs to be initialized.  
+- Enter the project home directory where you want hot reload;  
+- Running `fileboy init` will generate `filegirl. yaml` files in this directory.  
+- View `filegirl. yaml` and modify it to a configuration item suitable for your project.  
+- Run `fileboy`.  
+  
+If you define the `command-> exec` command, you can run the `fileboy exec` command to confirm whether it can be executed properly in advance. The system will try to run your custom command.  
+You can use `fileboy help` to see help info.  
 
 ## filegirl.yaml 配置文件说明
 
@@ -125,12 +153,75 @@ notifier:
     # 请求超时 15 秒
     # POST 格式:
     #    Content-Type: application/json;charset=UTF-8
-    #    User-Agent: FileBoy Net Notifier v1.7
+    #    User-Agent: FileBoy Net Notifier v1.9
     #    Body: {"project_folder":"/watcher-dirs","file":"test.go","changed":1546421173070433800,"ext":".go"}
     # 例: http://example.com/notifier/fileboy-listener
     # 不启用通知，请留空 ""
     callUrl: ""
 ```
+___  
+```yaml
+core:
+    # config version code
+    version: 1
+
+# monitor section
+monitor:
+    # directories to monitor
+    # test1       listen for the test1 directory in the project directory
+    # test1/test2 listen for the test1/test2 directory in the project directory
+    # test1,*     listen for the test1 directory in the project directory and all its subdirectories (recursion)
+    # .,*         listen for the project directory and all its subdirectories (recursion)
+    includeDirs:
+        - .,*
+
+    # Unmonitored directories
+    # .idea   ignore listening to .idea directory and all its subdirectories
+    exceptDirs:
+        - .idea
+        - .git
+        - .vscode
+        - node_modules
+        - vendor
+
+    # the suffix of the listener file, which changes the file to execute commands
+    # .go   file changes suffixed with .go execute commands
+    # .*    all file changes execute commands in the command
+    types:
+        - .go
+
+command:
+    # the files monitored have commands that change to be executed
+    # there can be multiple commands that will be executed in turn
+    # in case of interactive commands, allow external access to input
+    # variable placeholders are supported, and the actual values are replaced when the command is run:
+    #    {{file}}    (e.g: a.txt 、test/test2/a.go)
+    #    {{ext}}     (e.g: .go)
+    #    {{changed}} local timestamp for file updated(nanosecond,e.g 1537326690523046400)
+    # variable placeholders e.g：cp {{file}} /root/sync -rf  、 myCommand --{{ext}} {{changed}}
+    exec:
+        - go version
+        - go env
+
+    # the command will not execute until XX milliseconds after the file changes
+    # a change event (A) cancels execution if there is a new file change event (B) within the defined delay time (t).
+    # B and subsequent events are analogized in turn until event Z does not produce new events within t, and Z executes.
+    # reasonable setting of delay time will effectively reduce redundancy and duplicate task execution.
+    # If this feature is not required, set to 0
+    delayMillSecond: 2000
+
+notifier:
+    # file changes send requests to the URL (POST JSON text data)
+    # the timing of triggering the request is consistent with executing the command command
+    # timeout 15 second
+    # POST :
+    #    Content-Type: application/json;charset=UTF-8
+    #    User-Agent: FileBoy Net Notifier v1.9
+    #    Body: {"project_folder":"/watcher-dirs","file":"test.go","changed":1546421173070433800,"ext":".go"}
+    # e.g: http://example.com/notifier/fileboy-listener
+    # no notice is enabled. Please leave it blank. ""
+    callUrl: ""
+```
 
 ### TODO
 
@@ -142,8 +233,15 @@ notifier:
 - [x] 支持 http 通知  
 - [x] 支持冗余任务丢弃  
 - [ ] 支持 http 合并任务的通知  
-
-
+___  
+- [x] command supports variable placeholders  
+- [x] Supports multiple commands  
+- [x] Supports listening for specified folders  
+- [x] Supports not listening to specified folders  
+- [x] Supports listening for specified suffix files  
+- [x] Supports HTTP notifications  
+- [x] Supports redundant task discarding  
+- [ ] Notification supporting HTTP merge tasks  
 
 ## QA
 
@@ -154,7 +252,6 @@ notifier:
 #### fileboy 可以应用在那些具体的场景？  
 
 在开发中，我们很需要一款可以帮助我们自动打包编译的工具，那 fileboy 就非常适合这样的场景。比如 go 项目的热编译，让我们可以边修改代码边运行得到反馈。又比如 PHP Swoole 框架，由于常驻进程的原因，无法更改代码立即reload，使用 fileboy 就可以辅助做到传统 PHP 开发的体验。  
-
 对于一些需要监控文件日志或者配置变动的场景， fileboy 同样适合。你可以事先编写好相应的通知报警脚本，然后定义`filegirl.yaml`中的`command`命令，交由 fileboy 自动运行监控报警。  
 
 #### 通知器在什么时候会发送 http 请求 ?
@@ -168,17 +265,11 @@ notifier:
 #### filegirl.yaml 里面的 command 不支持复杂的命令吗？  
 
 对于“很复杂的命令”这种说法很难去定义，比如 `echo "hello world"`并不复杂，但是对于 fileboy 来讲，目前无法解析这种命令。  
-
 fileboy 目前支持 `命令 + 参数`这种形式的 command，而且 参数中不能有""符号或者有空格。如：  
-
 `go build`:支持；  
-
 `go env`:支持;  
-
 `php swoole start --daemon`:支持  
-
 `cat a.txt | grep "q" | wc -l`:不支持  
-
 对于不支持的命令，可以把它写到一个文件里，然后在 command 中执行这个文件来解决。  
 
 #### 为什么起名为 fileboy，又把配置名叫做 filegirl ？

fileboy.go

@@ -9,6 +9,7 @@ import (
 	"math/rand"
 	"os"
 	"path"
+	"strconv"
 	"strings"
 	"time"
 )
@@ -40,7 +41,9 @@ func parseConfig() {
 	cfg = new(FileGirl)
 	fc, err := ioutil.ReadFile(projectFolder + "/filegirl.yaml")
 	if err != nil {
-		log.Panicln(PreError, "read filegirl.yaml file err: ", err)
+		log.Println(PreError, "the filegirl.yaml file in", projectFolder, "is not exist! ", err)
+		fmt.Print(firstRunHelp)
+		log.Fatalln("fileboy unable to run.")
 	}
 	err = yaml.Unmarshal(fc, cfg)
 	if err != nil {
@@ -49,15 +52,22 @@ func parseConfig() {
 	if cfg.Core.Version > Version {
 		log.Panicln(PreError, "current fileboy support max version : ", Version)
 	}
+	// init map
+	cfg.Monitor.TypesMap = map[string]bool{}
+	cfg.Monitor.IncludeDirsMap = map[string]bool{}
+	cfg.Monitor.ExceptDirsMap = map[string]bool{}
+	// convert to map
+	for _, v := range cfg.Monitor.Types {
+		cfg.Monitor.TypesMap[v] = true
+	}
 	log.Println(cfg)
 }
 
 func eventDispatcher(event fsnotify.Event) {
 	ext := path.Ext(event.Name)
 	if len(cfg.Monitor.Types) > 0 &&
-		cfg.Monitor.Types[0] != ".*" &&
-		!inStringArray(ext, cfg.Monitor.Types) {
-		//log.Println(ext, cfg.Monitor.Types, inStringArray(ext, cfg.Monitor.Types))
+		!keyInMonitorTypesMap(".*", cfg) &&
+		!keyInMonitorTypesMap(ext, cfg) {
 		return
 	}
 	switch event.Op {
@@ -77,66 +87,72 @@ func eventDispatcher(event fsnotify.Event) {
 
 func addWatcher() {
 	log.Println("collecting directory information...")
-	dirs := make([]string, 0)
-	for i := 0; i < len(cfg.Monitor.IncludeDirs); i++ {
-		darr := dirParse2Array(cfg.Monitor.IncludeDirs[i])
+	dirsMap := map[string]bool{
+		projectFolder: true,
+	}
+	for _, dir := range cfg.Monitor.IncludeDirs {
+		darr := dirParse2Array(dir)
 		if len(darr) < 1 || len(darr) > 2 {
-			log.Fatalln(PreError, "filegirl section monitor dirs is error. ", cfg.Monitor.IncludeDirs[i])
+			log.Fatalln(PreError, "filegirl section monitor dirs is error. ", dir)
 		}
 		if strings.HasPrefix(darr[0], "/") {
-			log.Fatalln(PreError, "dirs must be relative paths ! err path:", cfg.Monitor.IncludeDirs[i])
+			log.Fatalln(PreError, "dirs must be relative paths ! err path:", dir)
 		}
 		if darr[0] == "." {
 			if len(darr) == 2 && darr[1] == "*" {
-				dirs = make([]string, 0)
-				dirs = append(dirs, ".")
+				// The highest priority
+				dirsMap = map[string]bool{
+					projectFolder: true,
+				}
 				listFile(projectFolder, func(d string) {
-					dirs = arrayUniqueAdd(dirs, d)
+					dirsMap[d] = true
 				})
+				break
 			} else {
-				dirs = arrayUniqueAdd(dirs, projectFolder)
+				dirsMap[projectFolder] = true
 			}
-			break
 		} else {
 			md := projectFolder + "/" + darr[0]
+			dirsMap[md] = true
 			if len(darr) == 2 && darr[1] == "*" {
-				dirs = arrayUniqueAdd(dirs, md)
 				listFile(md, func(d string) {
-					dirs = arrayUniqueAdd(dirs, d)
+					dirsMap[d] = true
 				})
-			} else {
-				dirs = arrayUniqueAdd(dirs, md)
 			}
 		}
 
 	}
-	for i := 0; i < len(cfg.Monitor.ExceptDirs); i++ {
-		p := projectFolder + "/" + cfg.Monitor.ExceptDirs[i]
-		dirs = arrayRemoveElement(dirs, p)
+	for _, dir := range cfg.Monitor.ExceptDirs {
+		if dir == "." {
+			log.Fatalln(PreError, "exceptDirs must is not project root path ! err path:", dir)
+		}
+		p := projectFolder + "/" + dir
+		delete(dirsMap, p)
 		listFile(p, func(d string) {
-			dirs = arrayRemoveElement(dirs, d)
+			delete(dirsMap, d)
 		})
 	}
-	for _, dir := range dirs {
+	for dir := range dirsMap {
 		log.Println("watcher add -> ", dir)
 		err := watcher.Add(dir)
 		if err != nil {
 			log.Fatalln(err)
 		}
 	}
+	log.Println("total monitored dirs: " + strconv.Itoa(len(dirsMap)))
 	log.Println("fileboy is ready.")
+	cfg.Monitor.DirsMap = dirsMap
 }
 
 func initWatcher() {
-	parseConfig()
 	var err error
+	if watcher != nil {
+		_ = watcher.Close()
+	}
 	watcher, err = fsnotify.NewWatcher()
 	if err != nil {
 		log.Fatalln(err)
 	}
-	defer watcher.Close()
-
-	done := make(chan bool)
 	taskMan = newTaskMan(cfg.Command.DelayMillSecond, cfg.Notifier.CallUrl)
 	go func() {
 		for {
@@ -155,22 +171,18 @@ func initWatcher() {
 		}
 	}()
 	addWatcher()
-	<-done
 }
 
 func parseArgs() {
-	l := len(os.Args)
-	if l == 1 {
-		_, err := ioutil.ReadFile(projectFolder + "/filegirl.yaml")
-		if err != nil {
-			log.Println(PreError, "the filegirl.yaml file does not exist! ", err)
-			fmt.Print(firstRunHelp)
-			return
-		}
+	switch len(os.Args) {
+	case 1:
+		parseConfig()
+		done := make(chan bool)
 		initWatcher()
+		defer watcher.Close()
+		<-done
 		return
-	}
-	if l == 2 {
+	case 2:
 		c := os.Args[1]
 		switch c {
 		case "init":
@@ -191,6 +203,8 @@ func parseArgs() {
 			fmt.Print(helpStr)
 		}
 		return
+	default:
+		log.Fatalln("Unknown parameters, use `fileboy help` show help info.")
 	}
 }
 

filegirl.go

@@ -8,6 +8,11 @@ type FileGirl struct {
 		Types       []string `yaml:"types"`
 		IncludeDirs []string `yaml:"includeDirs"`
 		ExceptDirs  []string `yaml:"exceptDirs"`
+		// convert to
+		TypesMap       map[string]bool `yaml:"-"`
+		IncludeDirsMap map[string]bool `yaml:"-"`
+		ExceptDirsMap  map[string]bool `yaml:"-"`
+		DirsMap        map[string]bool `yaml:"-"`
 	}
 	Command struct {
 		Exec            []string `yaml:"exec"`

mac-build-all

@@ -0,0 +1,12 @@
+export CGO_ENABLED=0
+export GOARCH=amd64
+export GOOS=darwin
+go build -ldflags "-s -w" -o fileboy-darwin-amd64.bin
+echo darwin ok
+export GOOS=linux
+go build -ldflags "-s -w" -o fileboy-linux-amd64.bin
+echo linux ok
+export GOOS=windows
+go build -ldflags "-s -w" -o fileboy-windows-amd64.exe
+echo windows ok
+

notifer.go

@@ -60,7 +60,7 @@ func (n *NetNotifier) dispatch(params *postParams) {
 		return
 	}
 	req.Header.Set("Content-Type", "application/json;charset=UTF-8")
-	req.Header.Set("User-Agent", "FileBoy Net Notifier v1.7")
+	req.Header.Set("User-Agent", "FileBoy Net Notifier v1.9")
 	resp, err := client.Do(req)
 	if err != nil {
 		log.Println(PreError, "notifier call failed. err:", err)

raw.go

@@ -61,7 +61,7 @@ notifier:
     # 请求超时 15 秒
     # POST 格式:
     #    Content-Type: application/json;charset=UTF-8
-    #    User-Agent: FileBoy Net Notifier v1.7
+    #    User-Agent: FileBoy Net Notifier v1.9
     #    Body: {"project_folder":"/watcher-dirs","file":"test.go","changed":1546421173070433800,"ext":".go"}
     # 例: http://example.com/notifier/fileboy-listener
     # 不启用通知，请留空 ""
@@ -104,13 +104,13 @@ var logo = `
  _____     _  | |      _____   ____)  ) |   | | |___| |
 |  ___)   | | | |     |  ___) |  __  (| |   | |\_____/ 
 | |      _| |_| |_____| |_____| |__)  ) |___| |  ___   
-|_|     (_____)_______)_______)______/ \_____/  (___)   V1.7
+|_|     (_____)_______)_______)______/ \_____/  (___)   V1.9
 `
 var statement = `Dengsgo [dengsgo@gmail.com] Open Source with MIT License`
 
 var versionDesc = `
- Version   fileboy: v1.7    filegirl: v` + strconv.Itoa(Version) + `
-Released   2019.1.24
+ Version   fileboy: v1.9    filegirl: v` + strconv.Itoa(Version) + `
+Released   2019.4.3
  Licence   MIT
   Author   dengsgo [dengsgo@gmail.com]
  Website   https://github.com/dengsgo/fileboy

util.go

@@ -6,13 +6,9 @@ import (
 	"strings"
 )
 
-func inStringArray(value string, arr []string) bool {
-	for _, v := range arr {
-		if value == v {
-			return true
-		}
-	}
-	return false
+func keyInMonitorTypesMap(k string, cfg *FileGirl) bool {
+	_, ok := cfg.Monitor.TypesMap[k]
+	return ok
 }
 
 func cmdParse2Array(s string, cf *changedFile) []string {
@@ -58,30 +54,10 @@ func listFile(folder string, fun func(string)) {
 	}
 }
 
-func arrayUniqueAdd(a []string, add string) []string {
-	if inStringArray(add, a) {
-		return a
-	}
-	return append(a, add)
-}
-
-func arrayRemoveElement(a []string, r string) []string {
-	i := -1
-	for k, v := range a {
-		if v == r {
-			i = k
-			break
-		}
-	}
-	if i == -1 {
-		return a
-	}
-	if len(a) == 1 && i == 0 {
-		return []string{}
-	}
-	return append(a[:i], a[i+1:]...)
-}
-
 func relativePath(folder, p string) string {
-	return strings.TrimPrefix(p, folder)
+	s := strings.ReplaceAll(strings.TrimPrefix(p, folder), "\\", "/")
+	if strings.HasPrefix(s, "/") && len(s) > 1 {
+		s = s[1:]
+	}
+	return s
 }