goclone

https://github.com/imthaghost/goclone.git

cloning crawler go golang website-cloner website-scraper

Go to file

Juan Mesaglio 7f0fcc9135 Merge pull request #48 from npalumbo/imthaghost/feature/support-serveport-config This change is to allow the serve port to be tweaked.		2023-06-18 23:45:52 -03:00
.github/workflows	switch to v20.04	2023-06-02 13:06:31 -04:00
cmd	Merge branch 'master' into imthaghost/feature/support-serveport-config	2023-06-17 20:05:21 +02:00
docs	update example	2022-03-23 20:35:16 -07:00
pkg	Remove strconv as suggested in code review	2023-06-17 20:03:53 +02:00
testutils	silence stdout in tests	2022-03-02 20:28:07 -03:00
.gitignore	update	2021-11-25 12:53:17 -08:00
.goreleaser.yml	update release process	2021-11-25 18:28:26 -08:00
LICENSE	init	2020-03-22 18:09:44 -07:00
README.md	Merge branch 'master' into imthaghost/feature/support-serveport-config	2023-06-17 20:05:21 +02:00
go.mod	Bump golang.org/x/crypto from 0.0.0-20191227163750-53104e6ec876 to 0.1.0	2023-06-17 23:47:32 +00:00
go.sum	Bump golang.org/x/crypto from 0.0.0-20191227163750-53104e6ec876 to 0.1.0	2023-06-17 23:47:32 +00:00

README.md

Copy websites to your computer! Goclone is a utility that allows you to download a website from the Internet to a local directory. Get html, css, js, images, and other files from the server to your computer. Goclone arranges the original site's relative link-structure. Simply open a page of the "mirrored" website in your browser, and you can browse the site from link to link as if you were viewing it online.

Official Website

Installation
- Brew
- Manual
Examples
Contributors

🚀 Installation

Brew

# tap
brew tap imthaghost/goclone
# install tool
brew install goclone

Manual

# Go version >= 1.20
go install github.com/imthaghost/goclone/cmd/goclone@latest

Or

# go get :)
go get github.com/imthaghost/goclone
# change to project directory using your GOPATH
cd $GOPATH/src/github.com/imthaghost/goclone/cmd/goclone
# build and install application
go install

Examples

# goclone <url>
goclone https://configtree.co

Usage

Usage:
  goclone <url> [flags]

Flags:
  -C, --cookie strings        Pre-set these cookies
  -h, --help                  help for goclone
  -o, --open                  Automatically open project in default browser
  -p, --proxy_string string   Proxy connection string. Support http and socks5 https://pkg.go.dev/github.com/gocolly/colly#Collector.SetProxy
  -s, --serve                 Serve the generated files using Echo.
  -P, --servePort int         Serve port number. (default 5000)
  -u, --user_agent string     Custom User Agent