GitBucket
4.21.2
Toggle navigation
Snippets
Sign in
Files
Branches
1
Releases
Issues
1
Pull requests
Labels
Priorities
Milestones
Wiki
Forks
mark.george
/
httpcat
Browse code
Add Content-Type filter to proxy mode
master
1 parent
3bb4438
commit
9c60ca0e172fa466d5db278ed469e5e22a58f4fc
Mark George
authored
on 18 Apr 2023
Patch
Showing
8 changed files
README.md
src/client.go
src/display.go
src/global_options.go
src/main.go
src/parsing.go
src/proxy.go
src/server.go
Ignore Space
Show notes
View
README.md
httpcat =============== A netcat-like tool for analysing or mocking HTTP requests. My first experiment with Go. Latest version: 2.4 ## Usage ``` httpcat <command> [OPTIONS] Commands: client Send an HTTP request proxy Start a reverse HTTP logging proxy server Start a mock HTTP server version Display version Command specific help: httpcat <command> --help ``` ### Global Options The following options apply to all commands. ``` -v, --verbose Show additional details. --headers Show headers and request/status line only (default is everything). --bodies Show bodies only (default is everything). --nocolour Don't use colours (default is requests in blue and responses in red). --notimestamps Don't show timestamps. --noindent Don't indent bodies (currently, only application/json bodies are indented). ``` ## Client Mode Sends an HTTP request to the specified server and displays the response. ### Usage ``` httpcat client [OPTIONS] URL ``` ### Client Specific Options ``` -H, --header A header to add to request in the form name|value. Use multiple times for multiple headers. -m, --method [GET|POST|PUT|DELETE|PATCH|HEAD|OPTIONS] HTTP method for request (default: GET). -b, --body Request body to send. ``` ### Example ``` httpcat client \ --method POST \ --body "Testing 123" \ --header "Content-Type|text/plain" \ --header "Authorization|Bearer abc123" \ http://localhost:8080/api/testing ``` Note that the `\` character in the above example is the line continuation character that allows you to break long lines into smaller lines in most Linux/macOS shells. For the Windows `cmd` shell, use `^` for line continuation. For PowerShell, use <code>`</code> (the backtick character). ## Server Mode Creates a web server with mock routes that listen on specific paths and return specific responses. Displays the details of any requests that it receives. ### Usage ``` httpcat server [OPTIONS] ``` ### Server Specific Options ``` -p, --port Port to listen on. (default: 8080). -r, --route Route which is made up of a path, a response body, a content type, and a response status, all separated by the pipe '|' character. Repeat for additional routes. -c, --cors Enable Cross Origin Resource Sharing (CORS). Allows all requested methods and headers. -H, --header A header to add to response in the form name|value. Use multiple times for multiple headers. ``` ### Example Linux/macOS: ``` httpcat server \ --port 8080 \ --route "/hello|Hello World|text/plain|200" \ --route "/goodbye|||204" \ --cors ``` Windows PowerShell: ``` httpcat server ` /port:8080 ` /route:"/hello|Hello World|text/plain|200" ` /route:"/goodbye|||204" ` /cors ``` This creates a web server that listens on port `8080` and responds to any requests to the `/hello` path with the response `Hello World` and response code `200`, and returns an empty response with a `204` response code to any requests made to `/goodbye`. The server will allow any requested headers and methods for CORS requests. ### Server Wildcard Mode If no routes are specified then the server will accept requests to all paths, returning a `200` response code with an empty body for `GET` requests and a `204` response code for all other request methods. ### Complex Bodies For mocking services with large/complex bodies, you can use shell command substitution to load the body for a route from a file: File `cust.json`: ``` { "data": { "id": "0266a95f-e57c-11ec-f37b-c20db6f2e117", "customer_code": "Boris-SVZ7", "first_name": "Boris", "last_name": "McNorris", "email": "boris@example.com", "customer_group_id": "0afa8de1-147c-11e8-edec-2b197906d816" } } ``` Linux/macOS: ``` httpcat server \ --port 8080 \ --route "/api/2.0/customers|$(cat cust.json)|application/json|201" \ --cors ``` Windows PowerShell: ``` httpcat server ` /port:8080 ` /route:"/api/2.0/customers|$(gc cust.json)|application/json|201" ` /cors ``` ## Proxy Mode Creates a reverse proxy that displays all HTTP requests and responses that pass through it. It will rewrite the port for redirect responses. ### Usage ``` httpcat proxy [OPTIONS] ``` ### Proxy Specific Options ``` -p, --port The port that the proxy listens on. -t, --target The URL for the target web server that the proxy forwards requests to. -f, --filter Only show bodies that match the given Content-Type. ``` ### Examples ``` httpcat proxy --port 8090 --target http://localhost:8080 # only show JSON bodies httpcat proxy --filter "application/json" --port 8090 --target http://localhost:8080 ``` Any requests sent to port 8090 will be forwarded to `http://localhost:8080` and all requests and responses will be displayed. Any redirect responses will be rewritten to use the specified port. Using the `filter` option will cause the bodies that don't match the given `Content-Type` to be suppressed, but the headers will still be shown. ## Building ``` cd src go build -o ../httpcat ``` ### Stripping ``` go build -ldflags "-s -w" -o ../httpcat ``` See the `link` docs for information on `ldflags`: https://pkg.go.dev/cmd/link ### Cross Compiling Linux ``` GOOS=linux GOARCH=amd64 go build -o ../httpcat-linux ``` Windows ``` GOOS=windows GOARCH=amd64 go build -o ../httpcat-win64.exe ``` Intel Mac ``` GOOS=darwin GOARCH=amd64 go build -o ../httpcat-mac-intel ``` M1/ARM64 Mac ``` GOOS=darwin GOARCH=arm64 go build -o ../httpcat-mac-arm64 ``` ## License Zero-Clause BSD License: https://opensource.org/license/0bsd/ Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted. THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
httpcat =============== A netcat-like tool for analysing or mocking HTTP requests. My first experiment with Go. ## Usage ``` httpcat <command> [OPTIONS] Commands: client Send an HTTP request proxy Start a reverse HTTP logging proxy server Start a mock HTTP server version Display version Command specific help: httpcat <command> --help ``` ### Global Options The following options apply to all commands. ``` -v, --verbose Show additional details. --headers Show headers and request/status line only (default is everything). --bodies Show bodies only (default is everything). --nocolour Don't use colours (default is requests in blue and responses in red). --notimestamps Don't show timestamps. --noindent Don't indent bodies (currently, only application/json bodies are indented). ``` ## Client Mode Sends an HTTP request to the specified server and displays the response. ### Usage ``` httpcat client [OPTIONS] URL ``` ### Client Specific Options ``` -H, --header A header to add to request in the form name|value. Use multiple times for multiple headers. -m, --method [GET|POST|PUT|DELETE|PATCH|HEAD|OPTIONS] HTTP method for request (default: GET). -b, --body Request body to send. ``` ### Example ``` httpcat client \ --method POST \ --body "Testing 123" \ --header "Content-Type|text/plain" \ --header "Authorization|Bearer abc123" \ http://localhost:8080/api/testing ``` Note that the `\` character in the above example is the line continuation character that allows you to break long lines into smaller lines in most Linux/macOS shells. For the Windows `cmd` shell, use `^` for line continuation. For PowerShell, use <code>`</code> (the backtick character). ## Server Mode Creates a web server with mock routes that listen on specific paths and return specific responses. Displays the details of any requests that it receives. ### Usage ``` httpcat server [OPTIONS] ``` ### Server Specific Options ``` -p, --port Port to listen on. (default: 8080). -r, --route Route which is made up of a path, a response body, a content type, and a response status, all separated by the pipe '|' character. Repeat for additional routes. -c, --cors Enable Cross Origin Resource Sharing (CORS). Allows all requested methods and headers. -H, --header A header to add to response in the form name|value. Use multiple times for multiple headers. ``` ### Example Linux/macOS: ``` httpcat server \ --port 8080 \ --route "/hello|Hello World|text/plain|200" \ --route "/goodbye|||204" \ --cors ``` Windows PowerShell: ``` httpcat server ` /port:8080 ` /route:"/hello|Hello World|text/plain|200" ` /route:"/goodbye|||204" ` /cors ``` This creates a web server that listens on port `8080` and responds to any requests to the `/hello` path with the response `Hello World` and response code `200`, and returns an empty response with a `204` response code to any requests made to `/goodbye`. The server will allow any requested headers and methods for CORS requests. ### Server Wildcard Mode If no routes are specified then the server will accept requests to all paths, returning a `200` response code with an empty body for `GET` requests and a `204` response code for all other request methods. ### Complex Bodies For mocking real web services it can be annoying to include a large body directly in the route definition. A better option is to use shell command substitution to load the body from a file: File `cust.json`: ``` { "data": { "id": "0266a95f-e57c-11ec-f37b-c20db6f2e117", "customer_code": "Boris-SVZ7", "first_name": "Boris", "last_name": "McNorris", "email": "boris@example.com", "customer_group_id": "0afa8de1-147c-11e8-edec-2b197906d816" } } ``` Linux/macOS: ``` httpcat server \ --port 8080 \ --route "/api/2.0/customers|$(cat cust.json)|application/json|201" \ --cors ``` Windows PowerShell: ``` httpcat server ` /port:8080 ` /route:"/api/2.0/customers|$(gc cust.json)|application/json|201" ` /cors ``` ## Proxy Mode Creates a reverse proxy that displays all HTTP requests and responses that pass through it. It will rewrite the port for redirect responses. ### Usage ``` httpcat proxy [OPTIONS] ``` ### Proxy Specific Options ``` -p, --port The port that the proxy listens on. -t, --target The URL for the target web server that the proxy forwards requests to. ``` ### Example ``` httpcat proxy --port 8090 --target http://localhost:8080 ``` Any requests sent to port 8090 will be forwarded to `http://localhost:8080` and all requests and responses will be displayed. Any redirect responses will be rewritten to use the specified port. ## Building ``` cd src go build -o ../httpcat ``` ### Stripping ``` go build -ldflags "-s -w" -o ../httpcat ``` See the `link` docs for information on flags: https://pkg.go.dev/cmd/link ### Cross Compiling Linux ``` GOOS=linux GOARCH=amd64 go build -o ../httpcat-linux ``` Windows ``` GOOS=windows GOARCH=amd64 go build -o ../httpcat-win64 ``` Intel Mac ``` GOOS=darwin GOARCH=amd64 go build -o ../httpcat-mac-intel ``` M1/ARM64 Mac ``` GOOS=darwin GOARCH=arm64 go build -o ../httpcat-mac-arm64 ``` ## License Zero-Clause BSD License Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted. THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
Ignore Space
Show notes
View
src/client.go
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License: https://opensource.org/license/0bsd/ */ package main import ( "fmt" "net/http" "os" "strings" ) type ClientCommand struct { Headers []string `short:"H" long:"header" description:"A header to add to request in the form name|value. Use multiple times for multiple headers."` Method string `short:"m" long:"method" choice:"GET" choice:"POST" choice:"PUT" choice:"DELETE" choice:"PATCH" choice:"HEAD" choice:"OPTIONS" description:"HTTP method for request." default:"GET"` Body string `short:"b" long:"body" description:"Request body to send."` Args struct { URL string } `positional-args:"yes" required:"yes"` } func (opts *ClientCommand) Execute(args []string) error { sendRequest(opts) return nil } func sendRequest(opts *ClientCommand) { client := http.Client{} request, _ := http.NewRequest(opts.Method, opts.Args.URL, strings.NewReader(opts.Body)) // add extra headers if specified if len(opts.Headers) > 0 { for _, header := range opts.Headers { if name, value, err := parseHeader(header); err == nil { if globalOptions.Verbose { fmt.Println("Adding request header - " + name + ": " + value) } request.Header.Add(name, value) } else { fmt.Println("The following header is not in the correct format: " + header + "\nCorrect format is: 'name|value'") os.Exit(1) } } } if response, err := client.Do(request); err != nil { fmt.Println("Could not connect to server") } else { defer response.Body.Close() showResponse(response) } }
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License */ package main import ( "fmt" "net/http" "os" "strings" ) type ClientCommand struct { Headers []string `short:"H" long:"header" description:"A header to add to request in the form name|value. Use multiple times for multiple headers."` Method string `short:"m" long:"method" choice:"GET" choice:"POST" choice:"PUT" choice:"DELETE" choice:"PATCH" choice:"HEAD" choice:"OPTIONS" description:"HTTP method for request." default:"GET"` Body string `short:"b" long:"body" description:"Request body to send."` Args struct { URL string } `positional-args:"yes" required:"yes"` } func (opts *ClientCommand) Execute(args []string) error { sendRequest(opts) return nil } func sendRequest(opts *ClientCommand) { client := http.Client{} request, _ := http.NewRequest(opts.Method, opts.Args.URL, strings.NewReader(opts.Body)) // add extra headers if specified if len(opts.Headers) > 0 { for _, header := range opts.Headers { if name, value, err := parseHeader(header); err == nil { if globalOptions.Verbose { fmt.Println("Adding request header - " + name + ": " + value) } request.Header.Add(name, value) } else { fmt.Println("The following header is not in the correct format: " + header + "\nCorrect format is: 'name|value'") os.Exit(1) } } } if response, err := client.Do(request); err != nil { fmt.Println("Could not connect to server") } else { defer response.Body.Close() showResponse(response) } }
Ignore Space
Show notes
View
src/display.go
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License */ package main import ( "bytes" "encoding/json" "fmt" "io/ioutil" "net/http" "net/http/httputil" "strings" "time" "github.com/fatih/color" ) func showRequest(req *http.Request) { showTimestamp() if !globalOptions.NoColour { color.Set(color.FgBlue) } if globalOptions.HeadersOnly { showRequestHeaders(req) } else if globalOptions.BodiesOnly { showRequestBody(req) } else { showRequestHeaders(req) fmt.Println() showRequestBody(req) } color.Unset() } func showFilteredRequest(req *http.Request) { showTimestamp() if !globalOptions.NoColour { color.Set(color.FgBlue) } if !globalOptions.BodiesOnly { showRequestHeaders(req) } color.Unset() fmt.Println("\n[Request body filtered]") } func showResponse(rsp *http.Response) { showTimestamp() if !globalOptions.NoColour { color.Set(color.FgRed) } if globalOptions.HeadersOnly { showResponseHeaders(rsp) } else if globalOptions.BodiesOnly { showResponseBody(rsp) } else { showResponseHeaders(rsp) fmt.Println() showResponseBody(rsp) } color.Unset() } func showFilteredResponse(req *http.Response) { showTimestamp() if !globalOptions.NoColour { color.Set(color.FgRed) } if !globalOptions.BodiesOnly { showResponseHeaders(req) } color.Unset() fmt.Println("\n[Response body filtered]") } func showRequestBody(req *http.Request) { // is there a body? if req.Body == nil || req.Body == http.NoBody { // nope, nothing to do return } originalBody, _ := ioutil.ReadAll(req.Body) // clone body since reading from request it will consume it copy1 := ioutil.NopCloser(bytes.NewBuffer(originalBody)) copy2 := ioutil.NopCloser(bytes.NewBuffer(originalBody)) // put one copy back into original request req.Body = copy1 // read body from other copy body, _ := ioutil.ReadAll(copy2) defer copy2.Close() if !globalOptions.NoIndent { if strings.Contains(req.Header.Get("Content-Type"), "application/json") { // if req.Header.Get("Content-Type") == "application/json" { var indentedJson bytes.Buffer err := json.Indent(&indentedJson, body, "", " ") if err == nil { // display indented version fmt.Println(&indentedJson) return } } } fmt.Println(string(body[:])) } func showResponseBody(rsp *http.Response) { // is there a body? if rsp.Body == nil || rsp.Body == http.NoBody { // nope, nothing to do return } originalBody, _ := ioutil.ReadAll(rsp.Body) // clone body since reading from request it will consume it copy1 := ioutil.NopCloser(bytes.NewBuffer(originalBody)) copy2 := ioutil.NopCloser(bytes.NewBuffer(originalBody)) // put one copy back into original request rsp.Body = copy1 // read body from other copy body, _ := ioutil.ReadAll(copy2) defer copy2.Close() if !globalOptions.NoIndent { if strings.Contains(rsp.Header.Get("Content-Type"), "application/json") { var indentedJson bytes.Buffer err := json.Indent(&indentedJson, body, "", " ") if err == nil { // display indented version fmt.Println(&indentedJson) return } } } fmt.Println(string(body[:])) } func showRequestHeaders(req *http.Request) { dumpBody := false dump, _ := httputil.DumpRequest(req, dumpBody) fmt.Println(strings.TrimSpace(string(dump[:]))) } func showResponseHeaders(rsp *http.Response) { dumpBody := false dumpRsp, _ := httputil.DumpResponse(rsp, dumpBody) fmt.Println(strings.TrimSpace(string(dumpRsp[:]))) } func showTimestamp() { if !globalOptions.NoTimestamps { fmt.Println(time.Now().Format(time.StampMilli)) } }
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License */ package main import ( "bytes" "encoding/json" "fmt" "io/ioutil" "net/http" "net/http/httputil" "strings" "time" "github.com/fatih/color" ) func showRequest(req *http.Request) { showTimestamp() if !globalOptions.NoColour { color.Set(color.FgBlue) } if globalOptions.HeadersOnly { showRequestHeaders(req) } else if globalOptions.BodiesOnly { showRequestBody(req) } else { showRequestHeaders(req) fmt.Println() showRequestBody(req) } color.Unset() } func showResponse(rsp *http.Response) { showTimestamp() if !globalOptions.NoColour { color.Set(color.FgRed) } if globalOptions.HeadersOnly { showResponseHeaders(rsp) } else if globalOptions.BodiesOnly { showResponseBody(rsp) } else { showResponseHeaders(rsp) fmt.Println() showResponseBody(rsp) } color.Unset() } func showRequestBody(req *http.Request) { // is there a body? if req.Body == nil || req.Body == http.NoBody { // nope, nothing to do return } originalBody, _ := ioutil.ReadAll(req.Body) // clone body since reading from request it will consume it copy1 := ioutil.NopCloser(bytes.NewBuffer(originalBody)) copy2 := ioutil.NopCloser(bytes.NewBuffer(originalBody)) // put one copy back into original request req.Body = copy1 // read body from other copy body, _ := ioutil.ReadAll(copy2) defer copy2.Close() if !globalOptions.NoIndent { if strings.Contains(req.Header.Get("Content-Type"), "application/json") { // if req.Header.Get("Content-Type") == "application/json" { var indentedJson bytes.Buffer err := json.Indent(&indentedJson, body, "", " ") if err == nil { // display indented version fmt.Println(&indentedJson) return } } } fmt.Println(string(body[:])) } func showResponseBody(rsp *http.Response) { // is there a body? if rsp.Body == nil || rsp.Body == http.NoBody { // nope, nothing to do return } originalBody, _ := ioutil.ReadAll(rsp.Body) // clone body since reading from request it will consume it copy1 := ioutil.NopCloser(bytes.NewBuffer(originalBody)) copy2 := ioutil.NopCloser(bytes.NewBuffer(originalBody)) // put one copy back into original request rsp.Body = copy1 // read body from other copy body, _ := ioutil.ReadAll(copy2) defer copy2.Close() if !globalOptions.NoIndent { if strings.Contains(rsp.Header.Get("Content-Type"), "application/json") { var indentedJson bytes.Buffer err := json.Indent(&indentedJson, body, "", " ") if err == nil { // display indented version fmt.Println(&indentedJson) return } } } fmt.Println(string(body[:])) } func showRequestHeaders(req *http.Request) { dumpBody := false dump, _ := httputil.DumpRequest(req, dumpBody) fmt.Println(strings.TrimSpace(string(dump[:]))) } func showResponseHeaders(rsp *http.Response) { dumpBody := false dumpRsp, _ := httputil.DumpResponse(rsp, dumpBody) fmt.Println(strings.TrimSpace(string(dumpRsp[:]))) } func showTimestamp() { if !globalOptions.NoTimestamps { fmt.Println(time.Now().Format(time.StampMilli)) } }
Ignore Space
Show notes
View
src/global_options.go
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License: https://opensource.org/license/0bsd/ */ package main type GlobalOptions struct { Verbose bool `short:"v" long:"verbose" description:"Show additional details."` HeadersOnly bool `long:"headers" description:"Show headers and request/status line only (default is everything)."` BodiesOnly bool `long:"bodies" description:"Show bodies only (default is everything)."` NoColour bool `long:"nocolour" description:"Don't use colours (default is requests in blue and responses in red)."` NoTimestamps bool `long:"notimestamps" description:"Don't show timestamps."` NoIndent bool `long:"noindent" description:"Don't indent bodies (currently, only application/json bodies are indented)."` } var ( globalOptions = new(GlobalOptions) )
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License */ package main type GlobalOptions struct { Verbose bool `short:"v" long:"verbose" description:"Show additional details."` HeadersOnly bool `long:"headers" description:"Show headers and request/status line only (default is everything)."` BodiesOnly bool `long:"bodies" description:"Show bodies only (default is everything)."` NoColour bool `long:"nocolour" description:"Don't use colours (default is requests in blue and responses in red)."` NoTimestamps bool `long:"notimestamps" description:"Don't show timestamps."` NoIndent bool `long:"noindent" description:"Don't indent bodies (currently, only application/json bodies are indented)."` } var ( globalOptions = new(GlobalOptions) )
Ignore Space
Show notes
View
src/main.go
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License: https://opensource.org/license/0bsd/ */ package main import ( "fmt" "os" "github.com/jessevdk/go-flags" ) const version string = "2.4" type VersionCommand struct{} func (c *VersionCommand) Execute(args []string) error { fmt.Println("httpcat v" + version) return nil } func main() { parser := flags.NewParser(globalOptions, flags.Default) parser.AddCommand("client", "Send an HTTP request", "", &ClientCommand{}) parser.AddCommand("server", "Start a mock HTTP server", "", &ServerCommand{}) parser.AddCommand("proxy", "Start a reverse HTTP logging proxy", "", &ProxyCommand{}) parser.AddCommand("version", "Display version", "", &VersionCommand{}) _, err := parser.Parse() if err != nil { parseErr := err.(*flags.Error) if parseErr.Type.String() == flags.ErrCommandRequired.String() { os.Stderr.WriteString("\n") parser.WriteHelp(os.Stderr) } } }
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License */ package main import ( "fmt" "os" "github.com/jessevdk/go-flags" ) const version string = "2.3" type VersionCommand struct{} func (c *VersionCommand) Execute(args []string) error { fmt.Println("httpcat v" + version) return nil } func main() { parser := flags.NewParser(globalOptions, flags.Default) parser.AddCommand("client", "Send an HTTP request", "", &ClientCommand{}) parser.AddCommand("server", "Start a mock HTTP server", "", &ServerCommand{}) parser.AddCommand("proxy", "Start a reverse HTTP logging proxy", "", &ProxyCommand{}) parser.AddCommand("version", "Display version", "", &VersionCommand{}) _, err := parser.Parse() if err != nil { parseErr := err.(*flags.Error) if parseErr.Type.String() == flags.ErrCommandRequired.String() { os.Stderr.WriteString("\n") parser.WriteHelp(os.Stderr) } } }
Ignore Space
Show notes
View
src/parsing.go
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License: https://opensource.org/license/0bsd/ */ package main import ( "errors" "strconv" "strings" ) var ( headers map[string]string = make(map[string]string) bodies map[string]string = make(map[string]string) contentTypes map[string]string = make(map[string]string) statuses map[string]int = make(map[string]int) ) func parseHeader(header string) (string, string, error) { parts := strings.Split(header, "|") if len(parts) < 2 { return "", "", errors.New("header is not in the correct format") } else { name := strings.TrimSpace(parts[0]) value := strings.TrimSpace(parts[1]) headers[name] = value return name, value, nil } } func parseRoute(route string) (string, error) { parts := strings.Split(route, "|") if len(parts) != 4 { return "", errors.New("route is not in the correct format") } else { path := parts[0] body := parts[1] contentType := parts[2] status := parts[3] code, err := strconv.Atoi(status) if err != nil { return "", errors.New("route is not in the correct format") } bodies[path] = body statuses[path] = code contentTypes[path] = contentType return path, nil } }
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License */ package main import ( "errors" "strconv" "strings" ) var ( headers map[string]string = make(map[string]string) bodies map[string]string = make(map[string]string) contentTypes map[string]string = make(map[string]string) statuses map[string]int = make(map[string]int) ) func parseHeader(header string) (string, string, error) { parts := strings.Split(header, "|") if len(parts) < 2 { return "", "", errors.New("header is not in the correct format") } else { name := strings.TrimSpace(parts[0]) value := strings.TrimSpace(parts[1]) headers[name] = value return name, value, nil } } func parseRoute(route string) (string, error) { parts := strings.Split(route, "|") if len(parts) != 4 { return "", errors.New("route is not in the correct format") } else { path := parts[0] body := parts[1] contentType := parts[2] status := parts[3] code, err := strconv.Atoi(status) if err != nil { return "", errors.New("route is not in the correct format") } bodies[path] = body statuses[path] = code contentTypes[path] = contentType return path, nil } }
Ignore Space
Show notes
View
src/proxy.go
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License: https://opensource.org/license/0bsd/ */ package main import ( "fmt" "net/http" "net/http/httputil" "net/url" "strconv" "strings" ) var proxyOpts *ProxyCommand type ProxyCommand struct { Port int `short:"p" long:"port" description:"The port that the proxy listens on." required:"true"` Target string `short:"t" long:"target" description:"The URL for the target web server that the proxy forwards requests to." required:"true"` Filter string `short:"f" long:"filter" description:"Only show bodies that match the given Content-Type."` } func (opts *ProxyCommand) Execute(args []string) error { fmt.Println("Proxying port " + strconv.Itoa(opts.Port) + " to " + opts.Target) proxyOpts = opts proxy, err := CreateProxy(opts.Target) if err != nil { panic(err) } http.HandleFunc("/", RequestHandler(proxy)) if err := http.ListenAndServe(":"+strconv.Itoa(opts.Port), nil); err != nil { panic(err) } return nil } func CreateProxy(target string) (*httputil.ReverseProxy, error) { url, err := url.Parse(target) // bad target URL if err != nil { return nil, err } proxy := httputil.NewSingleHostReverseProxy(url) OriginalDirector := proxy.Director proxy.Director = func(req *http.Request) { FixHeaders(url, req) if proxyOpts.Filter != "" { contentType, hasContentType := req.Header["Content-Type"] // show body if it matches filter, or has no content-type header if !hasContentType || contentType[0] == proxyOpts.Filter { showRequest(req) } else { showFilteredRequest(req) } } else { showRequest(req) } // send request to target OriginalDirector(req) } proxy.ModifyResponse = func(rsp *http.Response) error { // rewrite the port for redirect responses switch rsp.StatusCode { case 301, 302, 303, 307, 308: url, _ := url.Parse(rsp.Header["Location"][0]) newUrl := RewritePort(url, strconv.Itoa(proxyOpts.Port)) rsp.Header["Location"][0] = newUrl.String() } if proxyOpts.Filter != "" { contentType, hasContentType := rsp.Header["Content-Type"] // show body if it matches filter, or has no content-type header if !hasContentType || contentType[0] == proxyOpts.Filter { showResponse(rsp) } else { showFilteredResponse(rsp) } } else { showResponse(rsp) } return nil } proxy.ErrorHandler = func(rsp http.ResponseWriter, req *http.Request, err error) { fmt.Println("Error sending request to target server:") fmt.Println(" " + err.Error()) } return proxy, nil } func FixHeaders(url *url.URL, req *http.Request) { // TODO this may be necessary for proxying HTTPS (need to test) req.URL.Host = url.Host req.URL.Scheme = url.Scheme req.Host = url.Host } func RewritePort(url *url.URL, newPort string) url.URL { var newUrl string if url.Port() != "" { // port exists, so replace :port with :newport newUrl = strings.Replace(url.String(), ":"+url.Port(), ":"+newPort, 1) } else { // no port, so replace hostname with hostname:port newUrl = strings.Replace(url.String(), url.Hostname(), url.Hostname()+":"+newPort, 1) } rewrittenUrl, _ := url.Parse(newUrl) return *rewrittenUrl } func RequestHandler(proxy *httputil.ReverseProxy) func(http.ResponseWriter, *http.Request) { return func(w http.ResponseWriter, r *http.Request) { proxy.ServeHTTP(w, r) } }
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License */ package main import ( "fmt" "net/http" "net/http/httputil" "net/url" "strconv" "strings" ) var ( proxyOpts *ProxyCommand ) type ProxyCommand struct { Port int `short:"p" long:"port" description:"The port that the proxy listens on." required:"true"` Target string `short:"t" long:"target" description:"The URL for the target web server that the proxy forwards requests to." required:"true"` } func (opts *ProxyCommand) Execute(args []string) error { fmt.Println("Proxying port " + strconv.Itoa(opts.Port) + " to " + opts.Target) proxyOpts = opts proxy, err := CreateProxy(opts.Target) if err != nil { panic(err) } http.HandleFunc("/", RequestHandler(proxy)) if err := http.ListenAndServe(":"+strconv.Itoa(opts.Port), nil); err != nil { panic(err) } return nil } func CreateProxy(target string) (*httputil.ReverseProxy, error) { url, err := url.Parse(target) // bad target URL if err != nil { return nil, err } proxy := httputil.NewSingleHostReverseProxy(url) OriginalDirector := proxy.Director proxy.Director = func(req *http.Request) { FixHeaders(url, req) showRequest(req) // send request to target OriginalDirector(req) } proxy.ModifyResponse = func(rsp *http.Response) error { // rewrite the port for redirect responses switch rsp.StatusCode { case 301, 302, 303, 307, 308: url, _ := url.Parse(rsp.Header["Location"][0]) newUrl := RewritePort(url, strconv.Itoa(proxyOpts.Port)) rsp.Header["Location"][0] = newUrl.String() } showResponse(rsp) return nil } proxy.ErrorHandler = func(rsp http.ResponseWriter, req *http.Request, err error) { fmt.Println("Error sending request to target server:") fmt.Println(" " + err.Error()) } return proxy, nil } func FixHeaders(url *url.URL, req *http.Request) { // TODO this may be necessary for proxying HTTPS (need to test) req.URL.Host = url.Host req.URL.Scheme = url.Scheme req.Host = url.Host } func RewritePort(url *url.URL, newPort string) url.URL { var newUrl string if url.Port() != "" { // port exists, so replace :port with :newport newUrl = strings.Replace(url.String(), ":"+url.Port(), ":"+newPort, 1) } else { // no port, so replace hostname with hostname:port newUrl = strings.Replace(url.String(), url.Hostname(), url.Hostname()+":"+newPort, 1) } rewrittenUrl, _ := url.Parse(newUrl) return *rewrittenUrl } func RequestHandler(proxy *httputil.ReverseProxy) func(http.ResponseWriter, *http.Request) { return func(w http.ResponseWriter, r *http.Request) { proxy.ServeHTTP(w, r) } }
Ignore Space
Show notes
View
src/server.go
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License: https://opensource.org/license/0bsd/ */ package main import ( "fmt" "net/http" "os" "strconv" ) type ServerCommand struct { Port int `short:"p" long:"port" description:"Port to listen on." default:"8080"` Routes []string `short:"r" long:"route" description:"Add a route which is made up of a path, a response body, a content type, and a response status code, all separated by the pipe '|' character. Repeat for additional routes. Example '/|Testing 123|text/plain|200'."` Cors bool `short:"c" long:"cors" description:"Enable Cross Origin Resource Sharing (CORS). Allows all requested methods and headers."` Headers []string `short:"H" long:"header" description:"A header to add to response in the form name|value. Use multiple times for multiple headers."` } var ( serverOptions *ServerCommand ) func (opts *ServerCommand) Execute(args []string) error { serverOptions = opts // parse custom headers if provided if len(opts.Headers) > 0 { for _, header := range opts.Headers { if _, _, err := parseHeader(header); err != nil { fmt.Println("The following header is not in the correct format: " + header + "\nCorrect format is: 'name|value'") os.Exit(1) } } } fmt.Println("HTTP server listening on port " + strconv.Itoa(opts.Port) + "\n") if len(opts.Routes) > 0 { fmt.Println("Routes:") for _, route := range opts.Routes { if path, err := parseRoute(route); err == nil { fmt.Println(" " + path) http.HandleFunc(path, serverRequestHandler) } else { fmt.Println("The following route is not in the correct format: " + route + "\nCorrect format is: \"/path|response body|content type|status code\"") os.Exit(1) } } fmt.Println() } else { // wildcard mode http.HandleFunc("/", serverRequestHandler) } if err := http.ListenAndServe(":"+strconv.Itoa(opts.Port), nil); err != nil { fmt.Fprintf(os.Stderr, "Could not start server. Is port %d available?\n", opts.Port) os.Exit(1) } return nil } func serverRequestHandler(rsp http.ResponseWriter, req *http.Request) { path := req.RequestURI body := bodies[path] contentType := contentTypes[path] code := statuses[path] // are we in wildcard mode? if code == 0 && len(statuses) == 0 { if req.Method == http.MethodGet { code = 200 } else { code = 204 } } else if code == 0 { // not in wildcard mode so this is a bad path code = 404 } showRequest(req) // CORS preflight if serverOptions.Cors && req.Method == http.MethodOptions { // was the OPTIONS request actually a CORS request? if secFetchMode := req.Header.Get("Sec-Fetch-Mode"); secFetchMode == "cors" { if globalOptions.Verbose { fmt.Println("Responding to CORS preflight") } if origin := req.Header.Get("Origin"); origin != "" { if globalOptions.Verbose { fmt.Println("Adding CORS response header - Access-Control-Allow-Origin: " + origin) } // allow the origin rsp.Header().Set("Access-Control-Allow-Origin", origin) if headers := req.Header.Get("Access-Control-Request-Headers"); headers != "" { if globalOptions.Verbose { fmt.Println("Adding CORS response header - Access-Control-Allow-Headers: " + headers) } // allow any requested headers rsp.Header().Set("Access-Control-Allow-Headers", headers) } if method := req.Header.Get("Access-Control-Request-Method"); method != "" { if globalOptions.Verbose { fmt.Println("Adding CORS response header - Access-Control-Allow-Methods: " + method) } // allow any requested methods rsp.Header().Set("Access-Control-Allow-Methods", method) } if globalOptions.Verbose { fmt.Println("Adding CORS response header - Access-Control-Max-Age: " + "600") } // allow browser to reuse the permissions for 10 minutes rsp.Header().Set("Access-Control-Max-Age", "600") rsp.WriteHeader(200) return } else { if globalOptions.Verbose { fmt.Println("CORS preflight did not contain 'Origin' header!") } rsp.Header().Set("Content-Type", "text/plain") rsp.WriteHeader(400) fmt.Fprint(rsp, "Origin header was not provided") return } } } // add custom headers if provided if len(serverOptions.Headers) > 0 { for name, value := range headers { if globalOptions.Verbose { fmt.Println("Adding response header - " + name + ": " + value) } rsp.Header().Set(name, value) } } // add CORS allow-origin header if serverOptions.Cors && req.Method != http.MethodOptions { if origin := req.Header.Get("Origin"); origin != "" { if globalOptions.Verbose { fmt.Println("Adding CORS response header - Access-Control-Allow-Origin: " + origin) } rsp.Header().Set("Access-Control-Allow-Origin", origin) } } // add content type if specified if len(contentType) > 0 { rsp.Header().Set("Content-Type", contentType) } // add status line rsp.WriteHeader(code) // add body fmt.Fprint(rsp, body) }
/** Author: Mark George <mark.george@otago.ac.nz> License: Zero-Clause BSD License */ package main import ( "fmt" "net/http" "os" "strconv" ) type ServerCommand struct { Port int `short:"p" long:"port" description:"Port to listen on." default:"8080"` Routes []string `short:"r" long:"route" description:"Add a route which is made up of a path, a response body, a content type, and a response status code, all separated by the pipe '|' character. Repeat for additional routes. Example '/|Testing 123|text/plain|200'."` Cors bool `short:"c" long:"cors" description:"Enable Cross Origin Resource Sharing (CORS). Allows all requested methods and headers."` Headers []string `short:"H" long:"header" description:"A header to add to response in the form name|value. Use multiple times for multiple headers."` } var ( serverOptions *ServerCommand ) func (opts *ServerCommand) Execute(args []string) error { serverOptions = opts // parse custom headers if provided if len(opts.Headers) > 0 { for _, header := range opts.Headers { if _, _, err := parseHeader(header); err != nil { fmt.Println("The following header is not in the correct format: " + header + "\nCorrect format is: 'name|value'") os.Exit(1) } } } fmt.Println("HTTP server listening on port " + strconv.Itoa(opts.Port) + "\n") if len(opts.Routes) > 0 { fmt.Println("Routes:") for _, route := range opts.Routes { if path, err := parseRoute(route); err == nil { fmt.Println(" " + path) http.HandleFunc(path, serverRequestHandler) } else { fmt.Println("The following route is not in the correct format: " + route + "\nCorrect format is: \"/path|response body|content type|status code\"") os.Exit(1) } } fmt.Println() } else { // wildcard mode http.HandleFunc("/", serverRequestHandler) } if err := http.ListenAndServe(":"+strconv.Itoa(opts.Port), nil); err != nil { fmt.Fprintf(os.Stderr, "Could not start server. Is port %d available?\n", opts.Port) os.Exit(1) } return nil } func serverRequestHandler(rsp http.ResponseWriter, req *http.Request) { path := req.RequestURI body := bodies[path] contentType := contentTypes[path] code := statuses[path] // are we in wildcard mode? if code == 0 && len(statuses) == 0 { if req.Method == http.MethodGet { code = 200 } else { code = 204 } } else if code == 0 { // not in wildcard mode so this is a bad path code = 404 } showRequest(req) // CORS preflight if serverOptions.Cors && req.Method == http.MethodOptions { // was the OPTIONS request actually a CORS request? if secFetchMode := req.Header.Get("Sec-Fetch-Mode"); secFetchMode == "cors" { if globalOptions.Verbose { fmt.Println("Responding to CORS preflight") } if origin := req.Header.Get("Origin"); origin != "" { if globalOptions.Verbose { fmt.Println("Adding CORS response header - Access-Control-Allow-Origin: " + origin) } // allow the origin rsp.Header().Set("Access-Control-Allow-Origin", origin) if headers := req.Header.Get("Access-Control-Request-Headers"); headers != "" { if globalOptions.Verbose { fmt.Println("Adding CORS response header - Access-Control-Allow-Headers: " + headers) } // allow any requested headers rsp.Header().Set("Access-Control-Allow-Headers", headers) } if method := req.Header.Get("Access-Control-Request-Method"); method != "" { if globalOptions.Verbose { fmt.Println("Adding CORS response header - Access-Control-Allow-Methods: " + method) } // allow any requested methods rsp.Header().Set("Access-Control-Allow-Methods", method) } if globalOptions.Verbose { fmt.Println("Adding CORS response header - Access-Control-Max-Age: " + "600") } // allow browser to reuse the permissions for 10 minutes rsp.Header().Set("Access-Control-Max-Age", "600") rsp.WriteHeader(200) return } else { if globalOptions.Verbose { fmt.Println("CORS preflight did not contain 'Origin' header!") } rsp.Header().Set("Content-Type", "text/plain") rsp.WriteHeader(400) fmt.Fprint(rsp, "Origin header was not provided") return } } } // add custom headers if provided if len(serverOptions.Headers) > 0 { for name, value := range headers { if globalOptions.Verbose { fmt.Println("Adding response header - " + name + ": " + value) } rsp.Header().Set(name, value) } } // add CORS allow-origin header if serverOptions.Cors && req.Method != http.MethodOptions { if origin := req.Header.Get("Origin"); origin != "" { if globalOptions.Verbose { fmt.Println("Adding CORS response header - Access-Control-Allow-Origin: " + origin) } rsp.Header().Set("Access-Control-Allow-Origin", origin) } } // add content type if specified if len(contentType) > 0 { rsp.Header().Set("Content-Type", contentType) } // add status line rsp.WriteHeader(code) // add body fmt.Fprint(rsp, body) }
Show line notes below