Building and Running the Watcher

Continuing a conversation from the MobileCoin Chat Group:

I got the production.jss and then consensus-enclave.css, using the sigstruct value from the former. So I guess path to this consensus-enclave.css is to be used in the “Example Invocation” snipet?

I am fine to use the two test nodes, as listed. I thought I would first run the watcher node (followed mobilecoin/README.md at master · mobilecoinfoundation/mobilecoin · GitHub), in order to sync the MC db, it compiled a bunch, however eventually threw an error about missing SGX signatures. My CPU has SGX support, so I probably could run the “full-fledged” command but not sure what to fill in for “–watcher-db”. I’m probably just too lame to get it to work at this stage. I should better start a noob thread on the forum, rather than wiping my frustration in there :open_mouth:

I did use the “mob” env to build the binaries, should all the invocation commands for mobilecoind & watcher be also executed from that?? If so, that was a miss on y side…

You shouldn’t need to remain in the mob env to run the binaries - can you paste the commands you’re running to enter the mob env and then to build?

The watcher README is a little light on details re: the environment variables you need in order to build. You should also set the SGX_MODE=HW, IAS_MODE=PROD and CONSENSUS_ENCLAVE_CSS=$(pwd)/consensus-enclave.css, so your build command may look something like:

SGX_MODE=HW IAS_MODE=PROD CONSENSUS_ENCLAVE_CSS=$(pwd)/consensus-enclave.css cargo build --release -p mc-watcher --bin watcher

Then you should be able to run the resulting binary, as:

./target/release/mc-watcher  \
    --tx-source-url https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node1.test.mobilecoin.com/ \
    --tx-source-url https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node2.test.mobilecoin.com/ \
    --watcher-db /tmp/watcher-db

Thanks Drakeley,

I gave it a fresh start today and after some adjustments, mc-watcher compiled successfully:

# SGX_MODE=HW IAS_MODE=PROD CONSENSUS_ENCLAVE_CSS=$(pwd)/consensus-enclave.css cargo build --release -p mc-watcher --bin watcher
error: no bin target named `watcher`

	Did you mean `mc-watcher`?
# 
# SGX_MODE=HW IAS_MODE=PROD CONSENSUS_ENCLAVE_CSS=$(pwd)/consensus-enclave.css cargo build --release -p mc-watcher --bin mc-watcher
info: syncing channel updates for 'nightly-2020-07-01-x86_64-unknown-linux-gnu'
info: latest update on 2020-07-01, rust version 1.46.0-nightly (16957bd4d 2020-06-30)
info: downloading component 'cargo'
info: downloading component 'clippy'
...<SNIP>...
   Compiling mc-watcher v1.0.0 (/home/dactylion/Downloads/mobilecoin/watcher)
    Finished release [optimized] target(s) in 13m 09s
# 

Ie. it was needed to change --bin watcher to --bin mc-watcher and run the build with elevated privileges. At least I hope it wasn’t too much of a blind shot.

Then however, trying to run the binary failed:

$ ./mc-watcher --tx-source-url https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node1.test.mobilecoin.com/ --tx-source-url https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node2.test.mobilecoin.com/ --watcher-db /tmp/watcher-db
2021-02-22 14:39:39.969764157 UTC INFO mc-watcher started: { "GIT_COMMIT": "0597b81", "PROFILE": "release", "DEBUG": "false", "OPT_LEVEL": "3", "DEBUG_ASSERTIONS": "false", "TARGET_ARCH": "x86_64", "TARGET_OS": "linux", "TARGET_FEATURE": "adx,aes,avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,mmx,movbe,pclmulqdq,popcnt,rdrand,rdseed,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave,xsavec,xsaveopt,xsaves", "RUSTFLAGS": "?", "SGX_MODE": "HW", "IAS_MODE": "PROD" }, mc.app: mc-watcher, mc.module: mc_common::logger::loggers, mc.src: common/src/logger/loggers/mod.rs:244
error: Found argument '--tx-source-url' which wasn't expected, or isn't valid in this context

USAGE:
    mc-watcher [FLAGS] [OPTIONS] --sources-path <sources-path>

For more information try --help
$ 

Since the command “only” complained about an unsupported argument and not the SGX params, I tried to run mobilecoind instead but that didn’t work out either. I guess it’s a combination of my lameness and untested GitHub code snippets?

$ SGX_MODE=HW IAS_MODE=PROD CONSENSUS_ENCLAVE_CSS=$(pwd)/consensus-enclave.css \   MC_LOG=debug,rustls=warn,hyper=warn,tokio_reactor=warn,mio=warn,want=warn,rusoto_core=error,h2=error,reqwest=error \
    cargo run --release -p mobilecoind -- \
    --ledger-db /tmp/mobilecoin/ledger-db \
    --watcher-db /tmp/mobilecoin/watcher-db \
    --poll-interval 1 \
    --peer mc://node1.test.mobilecoin.com/ \
    --peer mc://node2.test.mobilecoin.com/ \
    --tx-source-url https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node1.test.mobilecoin.com/ \
    --tx-source-url https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node2.test.mobilecoin.com/ \
    --mobilecoind-db /tmp/mobilecoin/mobilecoind-db \
    --listen-uri insecure-mobilecoind://127.0.0.1:4444/
error: package `mobilecoind` is not a member of the workspace
$ 

Any help is more than welcome. Thank you!

Hey @dactylion

The watcher recently went through some changes and we forgot to update the README.

To try it out with testnet, please do the following:

  1. Create a sources.toml file:
[[sources]]
tx_source_url = "https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node1.test.mobilecoin.com/"
consensus_client_url = "mc://node1.test.mobilecoin.com/"

[[sources]]
tx_source_url = "https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node2.test.mobilecoin.com/"
consensus_client_url = "mc://node2.test.mobilecoin.com/"
  1. Start the watcher using this command:
SGX_MODE=HW IAS_MODE=PROD MC_LOG=debug,hyper=error,want=error,reqwest=error,mio=error,rustls=error \
    cargo run -p mc-watcher --bin mc-watcher -- \
    --sources-path sources.toml \
    --watcher-db /tmp/watcher-db

For mobilecoind, you are close - try changing -p mobilecoind to -p mc-mobilecoind

Good luck!

Thanks @eran,

I created sources.toml and tried again with the supplied command but it failed again with tens of pages of errors. I am only pasting below what I think are the relevand parts.

$ SGX_MODE=HW IAS_MODE=PROD MC_LOG=debug,hyper=error,want=error,reqwest=error,mio=error,rustls=error \
    cargo run -p mc-watcher --bin mc-watcher -- \
    --sources-path sources.toml \
    --watcher-db /tmp/watcher-db
   Compiling mbedtls-sys-auto v2.18.1 (https://github.com/mobilecoinofficial/rust-mbedtls.git?tag=mc-0.3#8cac1fd1)
   Compiling grpcio-sys v0.6.0 (https://github.com/jcape/grpc-rs?rev=2ad042e9e65ecb664a60e034d0a899a8760d81d3#2ad042e9)
   Compiling bulletproofs v2.0.0 (htt
   ...<SNIP>...
      Compiling mc-attest-enclave-api v1.0.0 (/home/dactylion/Downloads/mobilecoin/attest/enclave-api)
The following warnings were emitted during compilation:

warning: Signed enclave not provided, trying to sign it...
warning: Unsigned enclave not provided, trying to link a new one...

error: failed to run custom build command for `mc-consensus-enclave-measurement v1.0.0 (/home/dactylion/Downloads/mobilecoin/consensus/enclave/measurement)`

Caused by:
  process didn't exit successfully: `/home/dactylion/Downloads/mobilecoin/target/debug/build/mc-consensus-enclave-measurement-7b6f38b4081ddb5d/build-script-build` (exit code: 101)
--- stdout
cargo:rerun-if-env-changed=CONSENSUS_ENCLAVE_CSS
cargo:rerun-if-env-changed=CONSENSUS_ENCLAVE_UNSIGNED
...<SNIP>...
--- stderr
thread 'main' panicked at 'Could not find SGX libraries, check PKG_CONFIG_PATH variable: Failure { command: "\"pkg-config\" \"--libs\" \"--cflags\" \"libsgx_urts\" \"libsgx_urts >= 2.9.101.2\" \"libsgx_urts <= 2.9.101.2\"", output: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "Package libsgx_urts was not found in the pkg-config search path.\nPerhaps you should add the directory containing `libsgx_urts.pc\'\nto the PKG_CONFIG_PATH environment variable\nPackage \'libsgx_urts\', required by \'virtual:world\', not found\nPackage \'libsgx_urts\', required by \'virtual:world\', not found\nPackage \'libsgx_urts\', required by \'virtual:world\', not found\n" } }', build.rs:38:10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

warning: build failed, waiting for other jobs to finish...
warning: 26 warnings emitted

error: build failed
thread 'main' panicked at 'Failed to get consensus-enclave sigstruct from build: TrustedBuild', consensus/enclave/measurement/build.rs:90:10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

warning: build failed, waiting for other jobs to finish...
error: build failed
$ 

Maybe I need to rebuild the whole thing, if you mention there have been changes? Or clean up somehow?

I thought I would only move my MOB from FTX but it is getting increasingly frustrating :smiley:

It looks like you do not have the Intel SGX libraries installed on your machine.
If building for main net you can try attempting to get the CSS file by doing:

SIGSTRUCT_URI=$(curl -s https://enclave-distribution.prod.mobilecoin.com/production.json | grep sigstruct | awk '{print $2}' | tr -d \")
    curl -O https://enclave-distribution.prod.mobilecoin.com/${SIGSTRUCT_URI}

and then adding the environment variable CONSENSUS_ENCLAVE_CSS=$(pwd)/consensus-enclave.css (as documented in mobilecoin/MAINNET.md at master · mobilecoinfoundation/mobilecoin · GitHub)

Thanks @eran,

Sorry, I’ve been a bit busy and only got to it now but adding the SGX libs seems to have done the trick!

On my Manjaro Nibia/20.2.1 I installed aur/linux-sgx-driver-dkms-git using yay. I also had to google a bit and update the build files from this GitHub repo for the compilation to finish successfully and have all the DKMS headers loaded.

Then I was able to run mobilecoind sing the sources.toml as you instructed above and it started syncing the from AWS onto my external drive. I have ~1000 blocks (~300 MB) in so far and am wondering how many there are (and how big the DB will be)?

Once done, I will read about mobilecoind-json and eventually pull my MOB from FTX and celebrate.
I will also try the GUI wallet to get the full experience :slight_smile:

Thanks a lot!

@eran why would 1000 blocks be 300 MB? Afaik a block on-chain only requires around 1kb if it contains one tx (which is the case for most current blocks) - so only 1MB for 1k blocks unless I am crazy.

I am not sure if @dactylion is running mobilecoind of mc-watcher since only the watcher uses sources.toml at the moment but they said they are running mobilecoind. Testnet has ~152K blocks. Using mobilecoind I end up with a ledger-db of about 311MB for the 152000 blocks.

Its worth mentioning that the ledger db includes extra data in additional to the raw block data as such extra indexes for lookups, tx outs merkle tree, etc.

Hi @eran & @koe,

Sorry if I caused some confusion.

This is the command I executed and that is still running (I replaced the username and UUID):

SGX_MODE=HW IAS_MODE=PROD CONSENSUS_ENCLAVE_CSS=$(pwd)/consensus-enclave.css \
    MC_LOG=debug,rustls=warn,hyper=warn,tokio_reactor=warn,mio=warn,want=warn,rusoto_core=error,h2=error,reqwest=error \
    cargo run --release -p mc-mobilecoind -- \
    --ledger-db /run/media/dactylion/UUID/mobilecoin/ledger-db \
    --watcher-db /run/media/dactylion/UUID/mobilecoin/watcher-db \
    --poll-interval 1 \
    --peer mc://node1.test.mobilecoin.com/ \
    --peer mc://node2.test.mobilecoin.com/ \
    --tx-source-url https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node1.test.mobilecoin.com/ \
    --tx-source-url https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node2.test.mobilecoin.com/ \
    --mobilecoind-db /run/media/dactylion/UUID/mobilecoin/mobilecoind-db \
    --listen-uri insecure-mobilecoind://127.0.0.1:4444/

…with the last messages being:

2021-03-03 08:19:59.744229797 UTC INFO Archive block retrieved for "https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node1.test.mobilecoin.com/" 105204, mc.app: mobilecoind, mc.module: mc_watcher::watcher, mc.src: watcher/src/watcher.rs:104
2021-03-03 08:19:59.788370805 UTC DEBG Lowest next block to sync: 105205, Ledger block height 152961, is_behind true, mc.app: mobilecoind, mc.module: mc_watcher::watcher, mc.src: watcher/src/watcher.rs:303
2021-03-03 08:19:59.788556025 UTC DEBG watcher sync is_behind: true lowest next block to sync: 105205 vs ledger: 152961, mc.app: mobilecoind, mc.module: mc_watcher::watcher, mc.src: watcher/src/watcher.rs:314
2021-03-03 08:19:59.788672524 UTC DEBG Now syncing signatures from 105205 to Some(105215), mc.app: mobilecoind, mc.module: mc_watcher::watcher, mc.src: watcher/src/watcher.rs:157
2021-03-03 08:19:59.788796478 UTC DEBG Attempting to fetch block 105205 from https://s3-us-west-1.amazonaws.com/mobilecoin.chain/node2.test.mobilecoin.com/00/00/00/00/00/01/9a/0000000000019af5.pb, mc.app: mobilecoind, mc.module: mc_watcher::watcher, mc.src: watcher/src/watcher.rs:96
2021-03-03 08:19:59.792737149 UTC DEBG Getting last block from mc://node1.test.mobilecoin.com:443/, mc.app: mobilecoind, mc.module: mc_ledger_sync::network_state::polling_network_state, mc.src: ledger/sync/src/network_state/polling_network_state.rs:77
2021-03-03 08:19:59.792911222 UTC DEBG Getting last block from mc://node2.test.mobilecoin.com:443/, mc.app: mobilecoind, mc.module: mc_ledger_sync::network_state::polling_network_state, mc.src: ledger/sync/src/network_state/polling_network_state.rs:77
2021-03-03 08:19:59.942675832 UTC DEBG Last block reported by mc://node1.test.mobilecoin.com:443/: 152960, mc.app: mobilecoind, mc.module: mc_ledger_sync::network_state::polling_network_state, mc.src: ledger/sync/src/network_state/polling_network_state.rs:87
2021-03-03 08:19:59.969539237 UTC DEBG Last block reported by mc://node2.test.mobilecoin.com:443/: 152960, mc.app: mobilecoind, mc.module: mc_ledger_sync::network_state::polling_network_state, mc.src: ledger/sync/src/network_state/polling_network_state.rs:87
2021-03-03 08:19:59.969574597 UTC DEBG Polling finished, current results: {ResponderId("node1.test.mobilecoin.com:443"): Some(152960), ResponderId("node2.test.mobilecoin.com:443"): Some(152960)}, mc.app: mobilecoind, mc.module: mc_ledger_sync::network_state::polling_network_state, mc.src: ledger/sync/src/network_state/polling_network_state.rs:119

The directories on my external drive look like this – not much of an increase since I last checked:

44K	./mobilecoind-db
304M	./ledger-db
67M	./watcher-db
371M	.

So I am at 105205 of 152961 and then I’m ready to start playing around with the wallets?

Cheers!

Yes, it looks like you are fully synced with the TestNet - so you are ready to play with that.
Note that for MainNet you need to follow the instructions in mobilecoin/MAINNET.md at master · mobilecoinfoundation/mobilecoin · GitHub