#+TITLE: FILE-FINDER #+SUBTITLE: Quickly search and manipulate augmented file objects.

Warning: This library is currently experimental. While perfectly usable as is, the application programming interface is prone to change in the future.

<2024-08-30>: =file-finder= is on Ultralisp and is included in [[https://github.com/ciel-lang/CIEL/][CIEL]].

<2024-10-16>: =file-finder= is on Quicklisp, since the 2024-10 release.

Please report any issue or suggestion, including:

• Better function, variable, slot, class or package naming.
• Better function arguments.
• Filesystem issues.

• Features

Enable rapid file search, inspection and manipulation.

• A =file= class which embeds the path and metadata such as size, ctime/mtime/atime.

• Various path manipulation functions which supersede Common Lisp =pathname= related functions.

  Using =file= instead of =pathname= saves us from many pitfalls, for instance path with wildcards (such as [, *) are no longer special.

• =finder= and =finder*= which return a list of files matching predicates.

  =finder= is a convenience wrapper around =finder*=. The latter has more options.

• A =file-finder:syntax= readtable to enable the =#f"/path/to/file"= syntax, which mimicks =#p= for pathnames.

In practice, it mostly supersedes:

• Common Lisp pathnames (at least for existing files).
• =find= for recursive and programmable file search. Unlike =find=, =finder='s predicates are extensible.
• =du=
• =touch=

Note that FILE-FINDER is not meant to manipulate arbitrary paths of non-existing files. Consider using [[https://github.com/fourier/ppath][ppath]] instead.

• Portability

For now this is only tested on Unix-based systems. Help welcome if you need support for another system.

• File search and recursive listing

List all files in the current directory, recursively:

#+begin_src lisp (finder) ; => (#F"/co…/file-finder/LICENSE" ; #F"/co…/file-finder/ffprobe.lisp" ; #F"/co…/file-finder/file.lisp" ; #F"/co…/file-finder/file-finder.asd" ; #F"~/co…/file-finder/mediafile.lisp" ; ...)

;; Same, with a given root, without descending into hidden directories and ;; without descending more than one level: (finder* :root (file ".") :recur-predicates (list (complement #'hidden?) (depth< 2))) #+end_src

List files matching all the given predicates:

#+begin_src lisp (finder "fil" (extension= "lisp")) ; => (#F"/co…/file-finder/file.lisp") ;; Passing a string as a predicate specifier is equivalent to `path': (finder (path~ "fil") (extension= "lisp"))

;; Passing a pathname is equivalent to `path$' (match end of path). #+end_src

We include useful predicate or predicate generators you can complete against, see the list below.

Passing a list of predicate specifiers connects them with a logical =or=. In other words, it returns the files matching at least one of the predicate specifiers:

#+begin_src lisp (finder (list "fil" (extension= "asd"))) ;; this returns many results: ; => (#F"/co…/file-finder/file.lisp" #F"/co…/file-finder/file-finder.asd" #F"~/co…/file-finder/mediafile.lisp")

;; While this finds 1 file: (finder "fil" (extension= "asd")) #+end_src

To get the file names, use =path=:

#+begin_src lisp (mapcar #'path *) ; => "~/co…/file-finder/mediafile.lisp" (sans the #F reader macro) #+end_src

For more complex predicate list nesting, you can leverage =alexandria:disjoin= and =alexandria:conjoin=.

** Hidden files and directories

=file-finder= doesn't return hidden files by default (on Linux, files starting with ".") and doesn't visit hidden directories.

You can search in them by setting 2 parameters:

#+begin_src lisp ;; List hidden files, descend hidden directories. (let ((include-hidden-files t) (descend-hidden-directories t)) (finder)) #+end_src

** Exclude directories

The =node_modules= directory is excluded by default.

You can enlarge the list into =exclude-directories= (use pathnames with =#p"directory/"= instead of strings).

** List of predicates

Most predicates are functions that accept one or many strings as arguments. In that case, they return a lambda function, that receives the file object as argument. For example:

#+begin_src lisp (finder (lambda (file-object) (str:containsp "lisp" (path file-object)))) ;; get the path name from the file object. #+end_src

Some predicates do not take arguments, such as =executable?=.

It is possible to use predicates that don't take arguments.

In =predicates.lisp=, see:

• =path~=: matches when one of the path elements is contained in the file path.
  • =every-path~=: same checks on the file path, but uses a logical =and=.
• =path$=: matches when one of the path suffixes matches the file path.
• =name==: matches when one of the names matches the file name (case sensitive).
• =iname==: matches when one of the names matches the file name (case insensitive).
• =name~=: matches when one of the names is contained in the file basename (and not the whole path), case sensitive.
  • =every-name~=: same checks on the file basename, but uses a logical =and=.
• =iname~=: matches when one of the names is contained in the file, case insensitive.
• =depth<=: matches when the argument file is in a subdirectory of ROOT less deep than LEVEL.
• =elf-binary?= and =elf-library?=.

• Make inspectable file objects

#+begin_src lisp (file "file-finder.asd") ; => #F"~/co…/file-finder/file-finder.asd"

(inspect *) ; => The object is a STANDARD-OBJECT of type FILE-FINDER/FILE::FILE. 0. PATH: "/home/lisp-maintainers/projects/file-finder/file-finder.asd"

1. INODE: 3223494
2. LINK-COUNT: 1
3. SIZE: 1565
4. DISK-USAGE: 12288
5. CREATION-DATE: @2023-11-16T19:08:16.000000+01:00
6. MODIFICATION-DATE: @2023-11-16T19:08:16.000000+01:00
7. ACCESS-DATE: @2024-04-22T17:50:58.000000+02:00

;; Enable reader macro: (named-readtables:in-readtable file-finder:syntax) ; => #<NAMED-READTABLE READTABLE {1003035363}>

;; Now you can use the #f syntax: #f"file-finder.asd" ; => #F"~/co…/file-finder/file-finder.asd"

;; Recursive disk-usage, in bytes. (disk-usage #f".") ; => 1298432

;; Custom printer with abbreviations disabled: (setf print-abbreviation-threshold 0 print-abbreviate-home? nil print-size? t print-date? t) ; => #F"/home/ambrevar/common-lisp/file-finder/file-finder.asd 348 Feb 28 16:56" #+end_src

;; Set permissions

(setf (permissions #f"file-finder.asd") '(:user-read :user-write :group-read))

; => (:USER-READ :USER-WRITE :GROUP-READ)

• Familiar path manipulation functions

#+begin_src lisp (separator) ; => "/"

(current-directory) ; => #F"~/co…/file-finder/"

(extension #f"file-finder.asd") ; => "asd" (basename #f"../file-finder/file-finder.asd") ; => "file-finder.asd" (parent #f"file-finder.asd") ; => #F"~/co…/file-finder/" (relative-path #f"file-finder.asd" #f"..") ; => "file-finder/file-finder.asd

(file? #f"file-finder.asd") ; => T (directory? #f"file-finder.asd") ; => NIL (let ((f #f"file-finder.asd")) (delete-file f) (exists? f)) ; => NIL #+end_src

• Changelog and acknowledgements

This library was cloned from [[https://gitlab.com/ambrevar/fof/][fof]] (file-object finder) by @ambrevar. FOF is richer in that its file object also gives: user and group IDs, stats, file kind (regular, executable) and permissions in user-readable format (:user-read, :user-write etc).

We are most interested in the search features, hence the clone and the cleanup. We could re-include some of the removed features by relying on the newer [[https://github.com/Shinmera/file-attributes][file-attributes]] library. PR welcome.

We did the following changes in this fork.

** 2024-04

*** Removed Osicat dependency and related features

• removed dependency on osicat, and added dependency on [[https://github.com/shinmera/file-attributes/][file-attributes]]
• removed: getting the user, group, stats, permissions, of a file and the related finder predicates (user, group, kind, executable).

In fof, an object has these slots:

#+BEGIN_SRC text 0. PATH: "/home/lisp-maintainers/projects/file-finder/file-finder.asd"

1. INODE: 3223494
2. LINK-COUNT: 1
3. KIND: :REGULAR-FILE
4. SIZE: 1565
5. DISK-USAGE: 12288
6. USER-ID: 1000
7. GROUP-ID: 1000
8. CREATION-DATE: @2023-11-16T19:08:16.000000+01:00
9. MODIFICATION-DATE: @2023-11-16T19:08:16.000000+01:00
10. ACCESS-DATE: @2024-04-22T17:50:58.000000+02:00
11. PERMISSIONS: (:USER-READ :USER-WRITE :GROUP-READ :GROUP-WRITE :OTHER-READ)

#+end_src

In file-finder, these ones:

#+BEGIN_SRC text 0. PATH: "/home/vince/quicklisp/local-projects/fof/readme.org"

1. INODE: 0
2. LINK-COUNT: 0
3. KIND: :REGULAR-FILE
4. SIZE: 0
5. DISK-USAGE: 0
6. CREATION-DATE: @1970-01-01T01:00:00.000000+01:00 <--- 1970 here
7. MODIFICATION-DATE: @2024-04-24T13:45:38.000000+02:00
8. ACCESS-DATE: @2024-04-24T13:45:38.000000+02:00

#+end_src

• removed dependency on =hu.dwim.defclass-star=.

• removed dependency on =trivia=.

• changed from package-inferred systems to traditional system (list dependencies in the .asd)

• changed from one package per file to one package for the project: simplify symbols mangling, no need to import&reexport and a bit easier to type for the user.

• added: don't return hidden files by default and don't recur into hidden directories (typically, .git/) (merged unmerged MR #2 of FOF).

• added: exclude node_modules/ directory by default.

*** Removed libmagic/magicffi dependency and dropped support for mime-type and file encoding

• [[https://github.com/guicho271828/magicffi][magicffi]] is a FFI wrapper around [[https://en.wikipedia.org/wiki/File_(command)][libmagic]], a unix command to recognize the type of data contained in a file. It makes installation and distribution less straightforward, and is unix only. FOF used it to get the file mime type and file encoding, and to provide related filters.
  • see also [[https://github.com/Shinmera/trivial-mimes][trivial-mimes]]. It first does a simple association file type -> mime type, without looking at the file content. If that fails, it consults the =file= unix command.