{{Header}}
{{#seo:
|description=Development Notes about git and ways to stay up to date of source code changes
}}
{{intro|
Development Notes about git and ways to stay up to date of source code changes
}}
= Security =
Definitions:
* '''remote server''': Means cloud server and/or third-party hosted. Not under the full control of the project.
* '''release builds''': Binary images such as ISO, VM images, Debian packages.
Policy:
* '''Untrusted git remote servers''': Git remote servers, such as GitHub, are considered untrustworthy.
* '''Local release builds only''': No release artifacts are produced on any remote servers. All builds are done locally.
* '''Secure usage of CI''': There is continuous integration, but it is for testing only. It is a "throw-away CI" only. A build is created on a remotely hosted server, but the build results (images) are not used for anything. Nothing gets touched or downloaded from the CI or any other remote server.
* '''No write access for CI servers''': CI servers or web servers do not have write access to source code (git).
* '''Compromised git remote servers''': Even if there was malicious write access to git, it would be noticed. Before making builds, the source code is always fetched, diffed, and reviewed before local merge. Only locally merged project source code that has been reviewed is used to create release builds.
* '''Non-essential use of CI only''': CI is only used as a "bonus", for non-essential tasks. Such as additional code scanners that are otherwise unavailable, infeasible or insecure to run locally.
See also: [[Trust]]
= Repositories =
{{project_name_short}}'s code is organized into a number of individual project repos, and a "master" repo for the build system that includes the other repos as submodules. Examples of individual project repos are:
* [https://github.com/Kicksecure/tirdad tirdad]
* [https://github.com/Kicksecure/sdwdate sdwdate]
* [https://github.com/Kicksecure/security-misc security-misc]
The master build system repo for {{project_name_short}} is [https://github.com/{{project_name_short}}/derivative-maker derivative-maker]. Its primary purpose is to build all of the individual projects, and create Debian-based OS images (OVA, ISO, QCOW2, etc) with these packages pre-installed.
Individuals who are interested in building Kicksecure or auditing its code should clone the [https://github.com/{{project_name_short}}/derivative-maker derivative-maker] repository and its submodules. Detailed instructions on building Kicksecure can be found in [[Dev/Build_Documentation/VM]]. Individuals interested in contributing to individual Kicksecure projects should clone the repository for the specific project they wish to contribute to.
= Branches =
== Introduction ==
While the project is small adrelanos thinks it is best not to make a too difficult policy.
== Compare ==
For individual project repositories, version bumps occur on every commit with the message bumped changelog version. You can find the commits corresponding to the release of the versions you are interested in, and compare them using git diff or git difftool.
Comparing the changes between two releases of {{project_name_short}} is harder because of git limitations when working with submodules. Each release of {{project_name_short}} is tagged, allowing you to use git checkout --recurse-submodules in the derivative-maker repository to go to a particular point in {{project_name_short}}'s development. However, the only tool in git that can show the changes between two releases of {{project_name_short}} with submodule changes included is git diff --submodule=diff. git difftool --submodule=diff will not show you changes within submodules.
To view the output of git diff --submodule=diff in an easier-to-read format, you can use [https://github.com/megatops/PatchViewer/blob/ff5515e77c99bdad9741caebf49c87d0fbe9d1eb/PatchViewer.html PatchViewer] to visualize the changes shown by Git. To use this:
* Open https://github.com/megatops/PatchViewer/blob/ff5515e77c99bdad9741caebf49c87d0fbe9d1eb/PatchViewer.html in your browser.
* Download the HTML file using the download button in the upper-right corner of the page.
* Open the downloaded HTML file in your browser.
* In a terminal, navigate to the derivative-maker repository, and run git diff --submodule=diff old-ref new-ref > diff, replacing old-ref and new-ref as appropriate.
* In PatchViewer, click "Choose File", and select the diff file you just generated.
* The diff will be visualized for you as two side-by-side files.
Note that this strategy will not show detailed information about changes to binary files such as images.
== master ==
* Adrelanos's progress of work.
* Not calling it unstable, because the changes will most likely never make it unstable but it is just not tested if it still builds.
* Branching model, project readme, information, gpg...
== signed git tags ==
Releases will be tagged and gpg signed.
== Link to Source Code ==
https://github.com/{{project_name_short}}/derivative-maker
-----
= subscribe to code changes =
== git ==
Git clone [https://github.com/{{project_name_short}}/derivative-maker github].
A git specific work flow could be:
git fetch
every (few) day(s) and then git diff(tool), merge, etc.
== rss feed notification ==
https://github.com/{{project_name_short}}/derivative-maker/commits/master.atom
== manually in your browser ==
Check every now and then https://github.com/{{project_name_short}}/derivative-maker/commits/master.
-----
{{Anchor|grep {{project_name_short}} source code}}
{{Anchor|grep}}
{{Anchor|grep {{project_name_long}} source code}}
{{anchor|grep_Whonix_source_code}}
= Search the Source Code =
== Introduction ==
greping (search for keywords inside files) or finding files the {{project_name_short}} source code can be useful when looking for certain keywords or files. However, a common grep -r or find . will show lots of irrelevant search results due to changelog files, license files and git history. The following is a suggestions on how to securely acquire {{project_name_short}} source code as well as on greping or finding the {{project_name_short}} with relevant only relevant search results.
== Get the Source Code ==
=== Get the Signing Key ===
{{Get_Signing_Key}}
=== Get the Source Code ===
{{Build_Documentation_Get_Source}}
=== OpenPGP Verify the Source Code ===
{{OpenPGP Verify the Source Code}}
=== Choose Version ===
{{
Build Documentation Choose Version
|version={{VersionNew}}-stable
|extra=--recurse-submodules
}}
=== Check Git ===
{{Build Documentation Check Git
|version={{VersionNew}}-stable
}}
== Find Keywords ==
'''1.''' Create folder ~/bin.
{{CodeSelect|code=
mkdir -p ~/bin
}}
'''2.''' Create a file ~/bin/mygrep.
'''3.''' Paste the following content.
{{CodeSelect|code=
#!/bin/bash
set -x
grep \
--exclude=README.md \
--exclude=GPLv2 \
--exclude=GPLv3 \
--exclude=COPYING \
--exclude=changelog.upstream-old1 \
--exclude-dir=mnt \
--exclude-dir=qubes-src/linux-template-builder/mnt \
--exclude=changelog.upstream \
--exclude-dir=".git" \
--exclude-dir=chroot-debian \
--exclude-dir=chroot-buster "$@"
}}
'''4.''' Save.
'''5.''' Make executable.
{{CodeSelect|code=
chmod +x ~/bin/mygrep
}}
'''6.''' Reboot required.
'''7.''' You need to be inside the source code folder. If not already, change directory.
{{CodeSelect|code=
cd derivative-maker
}}
'''8.''' Search for a textual string (keyword).
For example grub-pc. Replace grub-pc with the actual keyword.
{{CodeSelect|code=
mygrep -r -i grub-pc
}}
== Find Files ==
'''1.''' Create folder ~/bin.
{{CodeSelect|code=
mkdir -p ~/bin
}}
'''2.''' Create a file ~/bin/myfind.
'''3.''' Paste the following.
{{CodeSelect|code=
#!/bin/bash
set -x
find \
"$@" \
-type f \
-not -iwholename '*.git*'
}}
'''4.''' Save.
'''5.''' Make executable.
{{CodeSelect|code=
chmod +x ~/bin/myfind
}}
'''6.''' You need to be inside the source code folder. If not already, change directory.
{{CodeSelect|code=
cd derivative-maker
}}
'''7.''' Search for a file name.
For example /etc/apparmor.d.
{{CodeSelect|code=
myfind . {{!}} grep --color -i /etc/apparmor.d
}}
-----
= Compiled Code =
Source code that requires complication, that gets compiled into binaries.
{{CodeSelect|code=
find . -name '*.c' -not -iwholename '*.git*'
}}
./sdwdate/usr/lib/sdwdate/sclockadj.c
./bindp/usr/lib/bindp.c
./kloak/src/main.c
./kloak/src/keycodes.c
./kloak/src/eventcap.c
= Put folder under Git Version Control =
[[Operating System Software and Updates]]
{{CodeSelect|code=
sudo apt update
}}
Install git.
{{CodeSelect|code=
sudo apt install --no-install-recommends git
}}
Unless you want to use git for pushing changes to remotes which you probably won't in a testing VM you can use the following git config without using any real names or pseudonyms. (These are the git suggested defaults. [
]
git commit -a -m .
*** Please tell me who you are.
Run
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
fatal: empty ident name (for <(null)>) not allowed
)
{{CodeSelect|code=
git config --global user.email "you@example.com"
}}
{{CodeSelect|code=
git config --global user.name "Your Name"
}}
Desalinize git in current folder or cd to any folder you want to put under git version control. Can even be done for /home/user folder which is very useful.
{{CodeSelect|code=
git init
}}
Add all files to be added for next commit.
{{CodeSelect|code=
git add -A
}}
Actually commit to git.
{{CodeSelect|code=
git commit -a -m .
}}
Check git status of that folder.
{{CodeSelect|code=
git status
}}
No changes registered yet to the folder since just now committed all to git.
On branch master
nothing to commit, working tree clean
Create a test file or do some activity such as starting a browser or e-mail client.
{{CodeSelect|code=
touch test-file
}}
Check again git status of that folder.
{{CodeSelect|code=
git status
}}
Now git will show what changed.
On branch master
Untracked files:
(use "git add ..." to include in what will be committed)
test-file
nothing added to commit but untracked files present (use "git add" to track)
= Goodies =
Optional. Just sharing. Like it or not.
* Bash completion can complete git commands and branch names.
* [https://web.archive.org/web/20200201201521/http://www.opinionatedprogrammer.com/2011/01/colorful-bash-prompt-reflecting-git-status/ Colorful git prompt.]
[https://web.archive.org/web/20220801180038/http://www.opinionatedprogrammer.com/post-img/git-prompt-screenshot.png screenshot]
= apt source =
You need to enable deb-src in /etc/apt/sources.list.d/derivative.list!
sudo apt update
Hit:1 tor+http://deb.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion buster InRelease
Hit:2 https://deb.qubes-os.org/r4.1/vm bullseye InRelease
Ign:3 tor+http://2s4yqjx5ul6okpp3f2gaunr2syex5jgbfpfvhxxbbjwnrsvbk5v3qbid.onion/debian buster InRelease
Hit:4 tor+http://5ajw6aqf3ep7sijnscdzw77t7xq4xjpsy335yb2wiwgouo7yfxtjlmid.onion buster/updates InRelease
Hit:5 tor+http://2s4yqjx5ul6okpp3f2gaunr2syex5jgbfpfvhxxbbjwnrsvbk5v3qbid.onion/debian buster Release
Ign:7 https://deb.debian.org/debian buster InRelease
Hit:8 https://security.debian.org buster/updates InRelease
Hit:9 https://deb.debian.org/debian buster Release
Hit:11 https://deb.debian.org buster InRelease
Reading package lists... Done
E: Failed to fetch tor+http://deb.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/dists/{{Stable project version based on Debian codename}}/InRelease Unable to find expected entry 'contrib/source/Sources' in Release file (Wrong sources.list entry or malformed file)
E: Failed to fetch https://deb.debian.org/dists/stable/InRelease Unable to find expected entry 'non-free/source/Sources' in Release file (Wrong sources.list entry or malformed file)
E: Some index files failed to download. They have been ignored, or old ones used instead.
The following messages can be ignored:
* "Unable to find expected entry 'contrib/source/Sources'" and
* "Unable to find expected entry 'non-free/source/Sources'"
There is nothing in these repositories and it does not limit functionality in any way. This is an inconvenient message but it will not be fixed since it was only raised once in that last five years.
apt source is fully functional.
apt source sandbox-app-launcher
Reading package lists... Done
Picking 'kicksecure-meta-packages' as source package instead of 'sandbox-app-launcher'
NOTICE: 'kicksecure-meta-packages' packaging is maintained in the 'Git' version control system at:
https://github.com/{{project_name_short}}/sandbox-app-launcher.git
Please use:
git clone https://github.com/{{project_name_short}}/sandbox-app-launcher.git
to retrieve the latest (possibly unreleased) updates to the package.
Need to get 48.9 kB of source archives.
Get:1 tor+http://deb.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion buster/main sandbox-app-launcher 3:7.9-1 (dsc) [5,513 B]
Get:2 tor+http://deb.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion buster/main sandbox-app-launcher 3:7.9-1 (tar) [37.8 kB]
Get:3 tor+http://deb.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion buster/main sandbox-app-launcher 3:7.9-1 (diff) [5,588 B]
Fetched 48.9 kB in 2s (19.4 kB/s)
dpkg-source: info: extracting sandbox-app-launcher in sandbox-app-launcher-7.9
dpkg-source: info: unpacking sandbox-app-launcher_7.9.orig.tar.xz
dpkg-source: info: unpacking sandbox-app-launcher_7.9-1.debian.tar.xz
-----
= Footnotes =
{{reflist|close=1}}
{{Footer}}
[[Category:Development]] [[Category:Design]]