Integrating npm front-end with Kallithea

[Mads Kiilerich]

On 11/08/2017 03:05 PM, Andrew Shadura wrote:
> On 8 November 2017 at 14:42, Mads Kiilerich <mads at kiilerich.com> wrote:
>> On 11/08/2017 01:44 PM, Andrew Shadura wrote:
>>> Well, I did submit a proof of concept using lesscpy two years ago, and
>>> that went nowhere.
>> Can you find a reference to it so we can re-evaluate?
> https://lists.sfconservancy.org/pipermail/kallithea-general/2015q2/000788.html

Thanks.

Something like that could indeed be used to generate our current 
style.css, which is based on less since a couple of weeks ago.

(Once conern, though: Compiling from less on runtime is great for 
development, but for actual installation of released versions, it should 
probably generate it earlier so it can be served as a static file from 
CDN (or heavily cached)).

Could you or someone extend this proposaal to also cover the front-end 
build changes done in 
https://kallithea-scm.org/repos/kallithea/pull-request/108/_/bootstrap_in_small_bites_v7?fulldiff=1 
(and also handle jQuery and all the other front-end dependencies). (That 
would also require all dependency source to somehow be made available.)

> I'm much less concerned about vendoring than about a huge dependency
> on npm and its infrastructure.

That is a good point.

But without the dependency on npm and its infrastructure, *we* would 
have to provide an alternative huge dependency and infrastructure. That 
also doesn't sound appealing.

If we don't depend on npm to provide source of our dependencies, then we 
have to provide them somehow.

Then, some kind of automatic vendoring could seem appealing. It could 
perhaps be something that put the dependencies in release tar-balls 
without having to put their source in our source repository.

"Vendoring" could perhaps also happen by automatically "caching" and 
shipping everything npm would download, thus allow npm to run entirely 
offline?

>>> I don't understand what's so difficult about Lessing a bunch of files
>>> and amalgamating them afterwards so that we absolutely have to use the
>>> upstream's build system.

That might be a crucial point. I think it might be prohibitively 
difficult to not use upstream build systems.

Packages might do different things in their build system. It is not just 
about lessing a few files. I think it will be a significant amount of 
work and very fragile and make dependency version updates hard.

> My concern about npm isn't only that it is a whole different
> programming language, but that the infrastructure around npm is too
> chaotic to be reasonable, there are too many indirect dependencies,
> too many moving parts, things change a lot and too often. It's not
> something I would like to base anything, to be honest.

That might be true. But still, pragmatically, it pretty much works. It 
will allow us to move faster. Saying no to it will slow us down.

> I think we definitely need to somehow vendor jQuery and other js
> libraries, and — absolutely — Bootstrap. We need to be able to
> reproducibly build it from the source, and npm not only doesn't make
> that easier, but complicates that task and makes our build system more
> fragile.


Personally, I agree it could be nice if we compiled all front-end 
dependencies "directly from source" without using npm. But we don't 
"have" to do it if we don't ship compiled front-end code.


Anyway, short term, what do you suggest we do so we can move forward 
with 
https://kallithea-scm.org/repos/kallithea/pull-request/108/_/bootstrap_in_small_bites_v7?fulldiff=1 
?

I suggest we temporarily move forward with this "easy" solution of 
building front-end using "plain" npm at install time.

Meanwhile, others can work on distributing all dependency source and 
come up with a better solution than building with npm. (And perhaps 
also, perhaps as an intermediate step, make sure we offer all necessary 
sources so we can compile front-end code "offline" with npm and 
distribute it.

/Mads